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Row Value Constructor 

—Itzik Ben-Gan 

Future versions of SQL Server would benefit from sup- 
port for the row value constructor feature, which makes 
code less verbose, more readable, and easier to maintain. 


The Smart DBA’s Guide to 
SQL Server Disaster Recovery, 
Part 2 

—Michelle A. Poolet and Becky Smith 

In the conclusion of this two-part series, we show you 
how to develop, deploy, and perform day-to-day mainte- 
nance of your SQL Server disaster-recovery plan. 


Using Table-Valued Parameters 
to Update Multiple Rows 
—Maitthew S. Harris 

With SQL Server 2008's table-valued parameters, you 
can pass a table as a parameter to a stored procedure to 
update multiple field values in thousands of files. And 
you can easily add this functionality to an application to 
let users update the transactions themselves, rather than 
sending a request to the DBA. 
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41 Securing Credit Card Data 


Through SQL Server 2008 
—Edmund X. DeJesus 

There’s a lot more to PCI DSS compliance than meets the 
eye. Find out what’s behind the credit card industry's data 
security requirements and how SQL Server can help you 
lock down vulnerable data. 


Get the latest business intel- 


ligence articles and news at 
www.sqlmag.com/categories/ 
category/business-intelligence.aspx. 

—Megan Keller, SQL Server content manager 


Those who can envision a plausible 
future that's brighter than today will 
earn the opportunity to lead. 

Ray Ozzie October 28, 2010 
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The world has changed. 
~W- You need to know about your SQL Servers, whenever, wherever. 
~W- You're available 24/7 but your home life matters. It matters a lot. 


~W- You're always carrying an internet-connected device. 


18 months ago, Red Gate started the biggest project in its 
history to create the future of SQL Server monitoring. 


Find out whether we've earned the opportunity to lead: 
www.thefutureofmonitoring.com 
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Will the Cloud Kill 
Virtualization? 


hen virtualization first came on the market, 

there was a lot of confusion as to what it 
was and whether it could really be used for produc- 
tion workloads. Hosted virtualization products, 
such as Microsoft Virtual Server, didn’t provide the 
best performance. However, over time virtualization 
technology matured and hypervisor-based virtualiza- 
tion products such as VMware’s ESX Server and 
Microsoft’s Hyper-V were able to provide perfor- 
mance that’s comparable to bare-metal installations. 
Nowadays, virtualization is a key component of most 
IT infrastructures, and many companies only want to 
implement new virtual servers and need to justify if 
the server isn’t going to be virtual. 

Cloud computing seems to be going through its 
incubation stage much as virtualization did in its early 
days. Some companies are testing the waters, but very 
few are ready to jump in. Here’s how Microsoft and 
VMware are approaching cloud computing. 


Microsoft’s and VMware’s Views on 
Virtualization 

The tie-in between virtualization and the cloud is 
nebulous. However, at this year’s VMworld it became 
clear that the two leaders in the virtualization space, 
VMware and Microsoft, had different approaches to 
virtualization and very different visions for the cloud. 
As for virtualization, VMware has always approached 
virtualization from a basic level where virtualization 
could be considered a part of the hardware. This 
approach has made them the clear leader in the vir- 
tualization market, and ESX Server is renowned for 
its robustness and reliability. Microsoft likes to think 
of virtualization as an attribute of the OS, and its new 
Hyper-V role makes it very convenient for organiza- 
tions to easily adopt virtualization. 


Different Cloud Strategies 

Microsoft’s and VMware's approaches to the cloud 
are very different. VMware is moving in the direction 
in which IT is a service. The company is promoting a 
hybrid approach in which organizations might use ser- 
vices residing in both private and public clouds. New 
products such as VMware’s vCloud Director enable 
administrators to provision and secure VM resources 
in both public and private clouds. For VMware, pub- 
lic cloud services are provided by partners such as 
BlueLock, COLT, SingTel, Terremark, and Verizon. 
Like you might well imagine, virtualization is still a 


core tenant for VMware’s cloud strategy. When you 
provision new services, you're essentially creating new 
virtual machines (VMs), either in the private cloud 
(on-premises) or the public cloud (an Internet service 
provider). 

For Microsoft, the cloud enables an organization 
to completely bypass the need for virtualization. 
With this strategy, if you're using Microsoft’s cloud 
resources, there’s no need to bother with virtualiza- 
tion. Virtualization might be used to provide cloud 
services, but Microsoft handles that as the cloud pro- 
vider. Customers using cloud services never have to 
deal with the complexities of virtualization or infra- 
structure at all for that matter. If this scenario comes 
to play out, the cloud will eventually kill off the need 
for virtualization and most private infrastructure. 

Of course, some of the reasons behind these dif- 
ferent stances come from each company’s respective 
position in the industry. VMware is primarily a virtu- 
alization vendor, and it quite naturally has a very vir- 
tualization-oriented focus on IT. Microsoft, however, 
is running a distant second in the virtualization race 
and it doesn’t look like an area in which the company 
is ever going to win. At the same time, Microsoft has 
been in the online services business for more than a 
decade already, and it has several massive data cen- 
ters around the globe. Microsoft is well positioned to 
provide cloud services, whereas VMware is not. In 
case you're wondering how Microsoft differentiates 
between its Azure and Online Services cloud offerings, 
the Azure products, such as Windows Azure and SQL 
Azure, provide a platform on which you can build 
your own applications. In contrast, Exchange Online 
and SharePoint Online are ready-to-go services that 
you can subscribe to and use right away. 


Approach the Cloud with Caution 
Will the cloud kill virtualization? From my perspec- 
tive the answer is no—at least not in the foreseeable 
future. Businesses have had too many experiences 
with Internet connection failures, and stories of 
cloud outages from even the mighty cloud purveyors 
such as Google and Microsoft are way too common. 
Businesses are cautious and with good reason. That 
said, no business area changes faster than technol- 
ogy. Just as virtualization took time to mature into a 
production-ready technology, the same can probably 
be said about the cloud today. SQL 
InstantDoc ID 128795 
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ENHANCED SECURITY 
FOR MICROSOFT SQL 
SERVER 2008 STARTS 
WITH THALES 


Encryption is a must for database security and PCI DSS 
compliance. Now you can enhance the database 
encryption features of Microsoft SAL Sever 2008 and 
2008 R2 with Thales Hardware Security Modules 
(HSMs). These hardened, tamper-resistant devices 
are security certified by the U.S. Government and 
used by thousands of organizations worldwide. 
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and easily to raise 
encryption to the highest ia 
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Best of all, Thales HSMs — J 
enable centralized management of 
encryption keys across multiple databases for 
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optimized operations and simplify PCI DSS compliance. 


— with no database or 
business application 
modification required. 


Complimentary White Paper 


For a limited time, visit www.thalesesec.com/microsoft 
to download the white paper: “Database Encryption and 


Key Management for Microsoft SQL Server 2008 and 
2008 R2.” 
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Intelligent Backups Revisited 


ber 2010, InstantDoc ID 125991), I presented an 

automated solution for backing up transaction 
logs. I’ve also developed a solution for automating 
differential backups of databases and a script that lets 
you quickly restore transaction logs from backups if 
the need should arise. With all three tools, backing up 
and restoring your SQL Server system is a snap. 


i n “Intelligent Transaction Log Backups” (Novem- 


The Differential Backup 
Solution 

Like the transaction log backup solution, the differen- 
tial backup solution uses an admin job and a worker 
job. The admin/worker job concept is simple. You 
schedule the admin job to run periodically. Each time 
the admin job executes, it creates or edits the worker 
job, then starts it. The worker job, which is updated 
every time the admin job executes, is responsible for 
performing the differential backups of databases. 

In the differential backup solution, the admin job 
is a SQL Server Agent job named Server Daily Main- 
tenance - Differentials. This job contains three steps: 

1. The admin job first creates the msdb.dbo 
.SDM_Cleanup table if it doesn’t exist. This table 
stores records on database backups that have 
been performed. Records are added to the SDM_ 
Cleanup table by the worker job. 

2. The admin job calls the mnt_CleanUp 
stored procedure. As its name suggests, this 
stored procedure performs cleanup operations, 
including deleting old database backups. It also 
deletes the associated records in SDM_Cleanup. 

3. The admin job uses the mnt_BackupDif- 
ferential stored procedure to create a SQL Server 
Agent job named Maintenance_BackupDifferen- 
tial. The admin job then executes Maintenance_ 
Backup Differential, which is a worker job. 


The worker job will contain one step for each 
database. (You can choose to backup all databases 
or just databases in Full Recovery mode.) For each 
database, the worker job first checks to see whether 
a full backup exists. If a full backup doesn’t exist, it 
performs one. If a full backup exists, it performs a 
differential backup. The backups can be performed 
using the SQL Server native format or LiteSpeed 
format. (LiteSpeed is a third-party utility from Quest 
Software.) The default backup format is SQL Server 
native, so LiteSpeed isn’t required for this solution. 

I created the SDM_Differentials_QuickSetup 
.sql script to quickly set up the differential backup 
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solution. When you run SDM_Differentials_Quick- 

Setup.sql, it creates the mnt_CleanUp and mnt_Back- 

upDifferential stored procedures and the Server Daily 

Maintenance - Differentials admin job. 

Note that the admin and worker jobs use xp_cmd- 
shell when they run. I put xp_cmdshell inside a 
wrapper. That way, if you typically have xp_cmdshell 
disabled, the jobs will enable it for a brief moment 
so that they can perform their tasks, then return 
xp_cmdshell to the disabled state. If company policy 
forbids you to use xp_cmdshell even when it’s in a 
wrapper, you won't be able to use this solution. 

After you run SDM_Differentials_QuickSetup 
.sql, you can change some settings in the Server Daily 
Maintenance - Differentials admin job if desired: 
e By default, the Server Daily Maintenance - 

Differentials admin job is scheduled to run 
every six hours. If desired, you can 
change the frequency in the New Job 
Schedule page. 

e By default, the mnt_CleanUp stored pro- 
cedure is set to delete database backup 
files that are older than one day. If you 
want to change that default, find the code 


EXEC msdb.dbo.mnt_CleanUp 
@DaysOld = 1 


in the second job step and replace | with an inte- 
ger that represents the number of days you want 
the database backup files to be kept on the local 
server. Note that the stored procedure deletes 
only those backup files created by the transaction 
log backup solution. 

e By default, the worker job performs differ- 
ential backups of databases in Full Recovery 
mode using the SQL Server native format. 

It stores the backup files in E\MSSQL\DIF, 
overwriting any existing backup file for a 
database. If you want to change these defaults, 
find the code 


EXEC dbo.mnt_BackupDifferential 
@mode = 'Full', 
@WriteOver = 1, 
@Format = 'NV', 
@dir = N'E:\MSSQL\DIF' 


in the third job step. If you want to perform 
differential backups of all the databases, specify 
‘All’ for @mode. If you don’t want to overwrite 
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existing backup files, specify 0 for @WriteOver. If 
you want to use the LiteSpeed format, specify ‘LS’ 
for @Format. If you want to store the backup 
files in a different location, replace E\MSSQOL\ 
DIF with the desired directory in the @dir argu- 
ment. The directory you specify must exist. The 
worker job will create subdirectories in that direc- 
tory as needed, so the SQL Server service account 
must have write permissions to that directory. 


The differential backup solution is a useful 
supplement to the transaction log backup solution. 
Another helpful supplement is the RestoreLogs.sql 
script. 


RestoreLogs.sql 

When you use the transaction log backup solution, 
you can conceivably have hundreds of transaction log 
backup files, many of which might have been created 
after different database backups. This can make it 
difficult to determine which transaction log backups 
go with which database backup. RestoreLogs.sql can 
solve this problem for you. It identifies which trans- 
action log backups go with the latest full database 
backup and the order in which to restore the transac- 
tion log backups. And perhaps most important, it 
builds the restore commands for you. 

To use RestoreLogs.sql, the first thing you need to 
do is locate the latest full backup file and obtain the 
FirstLSN, which is the log sequence number (LSN) of 
the first log record in the backup set. For data and dif- 
ferential backups, the FirstLSN identifies the earliest 
log record that’s needed to perform a recovery. (You 
can read more about LSNs in the “Log Sequence 
Numbers and Restore Planning” web page at msdn 
-microsoft.com/en-us/library/ms190729.aspx.) 

To find the FirstLSN for SQL Server native back- 
ups, you can use the command 


RESTORE HEADERONLY 
FROM DISK = <insert path here> 


where <insert path here> is a path to the latest full 
backup file, such as N’E:\MSSQL\BAK \TestDB. bak’. 
For LiteSpeed backups, you can use the command 


EXEC master.dbo.xp_restore_headeronly 
@filename = <insert path here> 


After you have the FirstLSN, find the line 

SET @FirstLSN = 19990990005999099 

in RestoreLogs.sql and replace 19000000005900099 
with that number. 


Next, you need to specify where the transaction 
log backups are stored. Find the code 


SET @backupdirectory = 
N'E:\MSSQL\TRAN\Test ' 


and replace E:\\MSSQL\TRAMN\Test with your 
location. 

Finally, you need to specify the name of the data- 
base you want to restore. Find the line 


SET @DatabaseName = N'Test' 


and replace Test with your database’s name. 

Once these modifications are made, you can run 
the script. The results will not only show the transac- 
tion log backups used but also give the appropriate 
restore commands. 

RestoreLogs.sql is designed to work with the 
transaction log backups created by the transaction 
log backup solution. However, you can still use the 
script if you have an existing set of transaction log 
backups created by another process. You have two 
choices: Make the filenames compliant, or revise the 
code in RestoreLogs.sql. 

For the first option, you need to make sure that 
the names of your transaction log backup files follow 
two conventions: 

e The database’s name must be in the filename. 

e The filename needs to end in either __NV.trn 
for native backups or _LS.trn for LiteSpeed 
backups. 


For the second option, you need to modify 
and/or exclude the filters described in the first 
option. This isn’t difficult, but there’s insufficient 
space to go into details here. I recommend using 
the first option if you have a handful of transac- 
tion log backup files and the second option when 
renaming the files would become too tedious and 
time-consuming. 


A Complete Backup System 
With the differential backup solution and the 
transaction log backup solution, you have a 
complete backup system that you can set and 
forget. And when you need to do a restore, 
RestoreLogs.sql can quickly find the appro- 
priate transaction log backups and build the 
restore commands you need to use. You can 
download SDM_Differentials_QuickSetup.sql 
and RestoreLogs.sql by going to www.sqlmag 
.com, entering 128818 in the InstantDoc ID text 
box, clicking Go, then clicking the Download the 
Code Here button. These scripts work on SQL 
Server 2008 and SQL Server 2005. (To down- 
load the script that creates the admin job for the 
transaction log backup solution, see “Intelligent 
Transaction Log Backups.”) SQL 
InstantDoc ID 128818 
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SQL Server 2008 Spatial Tools 


Two executables that simplify how you work 


with spatial data 


Ni that SQL Server supports geometry and 
geography data types natively in the SQL 
Server 2008 (and later releases) relational engine, 
developers and DBAs are putting those data types 
to good use. There are many useful applications for 
spatial data types, and if you aren’t currently using 
them, you probably will in the near future. If you’ve 
never seen a well-crafted application using spatial 
data, I encourage you to take a look at toxicrisk.com, 
a website that shows EPA-registered toxic waste sites 
across the United States. (I was surprised to find that 
there are two such sites in my suburban home town.) 

We've been able to store longitude and latitude 
data since the inception of relational databases. But 
the beauty of spatial data types comes from their 
ease of use. Just because spatial data types are easier 
to use and more functional than older methods of 
storing geometric and geographic data, doesn’t mean 
it’s all strawberries and cream. For example, loading 
shape files (under the ESRI standard), can be time 
intensive and require a lot of experimentation to get 
it right. That’s where SQL Server 2008 Spatial Tools 
(SQLSpatialTools) comes in. 


Features 

Written by geographic information systems (GIS) 

coding enthusiast Morten Nielsen, SQLSpatialTools 

is made up of two simple executables that handle 

SQL Server spatial data: Shape2SQL, which enables 

you to load shape files into SQL Server 2008 and 

later, and the SQLSpatial Query Tool, which pulls 

spatial data out of SQL Server. Some key features of 

Shape2SQL include 

e Support for single point, multipoint, polygon, 
and linestring shape types 

e Geometry and geography data types 

e Schema object handling, such as replacing exist- 
ing tables and creating spatial indexes 


Some key features of the SQLSpatial Query Tool 

include 

e Using SQL queries to display the spatial results 
on a map 

e Displaying spatial data type attributes when 
hovering over geometry features on the map 
projection 

e Flexible renderings allowing thematic and cus- 
tom background maps 
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These tools don’t cover every imaginable use 
case for working with spatial data types. However, 
they'll get you up and running quickly and make this 
newer SQL Server feature much more accessible and 
useful. 


System Requirements 

SQLSpatialTools has rather modest requirements. 
Because spatial data types became available in SQL 
Server 2008, you'll naturally need to connect to 
SQL Server 2008 and later. The tools also require 
the .NET Framework 3.5. And because SQLSpa- 
tialTools use SQL Server CLR, you'll need either 
a full install of the SQL Server 2008 client or the 
Microsoft SQL Server System CLR Types on the 
client machine on which SQLSpatialTools run. 
Download SQLSpatialTools at www.sharpgis.net/ 
page/SQL-Server-2008-Spatial-Tools.aspx. 


Additional Free Resources 
There are other free tools you can use when doing 
spatial and mapping applications. Take a look at 
SharpMap on CodePlex (sharpmap.codeplex.com) 
for an easy-to-use mapping library written in C#. 
And if you want data, shape files, maps, and other 
geographic information, this is one area in which 
the United States and Canadian governments excel. 
For example, you can get almost all US public data 
in shape formats, such as the cartographic bound- 
ary files from each US census (www.census.gov/geo/ 
www/cob/bdy_files.html). I find that the MapCruzin 
website (www.mapcruzin.com/aboutus.htm) is a great 
clearinghouse for vast amounts of free data. SQL 
InstantDoc ID 128851 


SQL SERVER 2008 SPATIAL 
TOOLS 


Benefits: SQLSpatialTools makes the difficult 
job of loading and extracting spatial data quick 
and painless. 


System Requirements: SQL Server 2008 
or later; .NET Framework 3.5; SQL CLR via 
the SQL Server 2008 client or Microsoft SQL 
Server System CLR Types 


How to Get It: You can download SQLSpatial- 
Tools from www.sharpgis.net/page/SQL-Server- 
2008-Spatial-Tools.aspx. 


Kevin Kline 


(kevin.kline@ quest.com) is the director 


of technology for SQL Server Solutions 
at Quest Software and a founding board 
member of the international PASS. He 

is the author of SQL in a Nutshell, 3rd 
edition (O'Reilly). 
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GrllServer | SQL Server Magazine 
010 | Editors’ Best Awards 


Company: Confio Software 
Category: Best Database 
Monitoring/Performance Product 
Editors'Best} Product: Ignite 8 
GOED | Award: Gold 


Learn more about Ignite 8 at www.confio.com + 866.CONFIO.1 


2010 EDITOR'S SOL Server Magazine Gold Medal Winner 
BEST AWARD in Database Monitoring and Performance 


Only Ignite delivers a complete picture of database 
performance from an end user perspective. It analyzes 


response time, queries, sessions, and server resources 


A Editors’ Best 


to show both historical and real-time performance. 


Go to www.Confio.com and try Ignite 8 today! CONFIO” ohore. 


Boulder Colorado. 


Subscribe Now to the New 


Jev “roConnections 


(formerly asp.netPRO) 


WWW.deVprOconnecnians.Co: Bo wres : , l 
i| para DevProConnections magazine provides 


The Independent Resource for Microsoft Developers 


comprehensive, independent how-to 

content on building applications with 
J 

lak es the Stage the Microsoft stack — Visual Studio, 


ew i 4 .NET Framework, Azure, Windows, 


Windows Server, Office, SharePoint, 


My INET $ Story 


SQL Server, Mobile, and beyond. 


Pu 


e Ready-to-run code and pages e Monthly interviews with experts 


e Monthly guest editorial drilling in the development community 


down on a specific technology such e Columns on insider information 
as Silverlight or AJAX at Microsoft 


*$44 99 in Canada; $79.99 in all other countries 


The Cause of Fragmentation 
from Using Snapshot Isolation 


e’ve recently started to use snapshot 
isolation as a way to avoid blocking 
problems during our daily updates. This 


strategy is working well, but we’ve found that one 
table with a clustered index is becoming fragmented. 
The updates are affecting only fixed-width columns, 
so why is fragmentation happening? 


This is a tactic that we’ve used with a client to allow 
data loading into a database while concurrent que- 
ries are running. By enabling snapshot isolation just 
before the data loading commences, and switching it 
off again after the data loading finishes, the locks held 
by the data load don’t block the concurrent queries. 

The most well-known side effect of snapshot iso- 
lation is the extra load placed on tempdb. Snapshot 
isolation works by creating a chain of one or more 
prior versions of the record being changed. Each 
version of the record has an associated timestamp 
indicating the time at which that version was created, 
as well as a pointer to the location of the previous ver- 
sion of the record in the version store, which is located 
in tempdb. 

In a nutshell, whenever a record is changed, the 
previous version is copied into the version store. When 
a concurrent query needs to read a record, the storage 
engine traverses the record’s version chain to find the one 
that existed at the time the query started. The write and 
read activity accounts for the extra load on tempdb. 

The little-known side effect of snapshot isolation 
is the possibility of index fragmentation where you 


wouldn’t expect it, as you’ve explained you're seeing. 
Whenever a record 1s updated and a version is created, 
the new record must have a 14-byte tag appended to 
it. The tag holds the versioning timestamp and the 
pointer to the previous version of the record. 

When this operation occurs, the record obviously 
becomes 14 bytes longer, even if the only updated col- 
umns in the record are fixed-width (such as an integer 
or float column). For records stored in an index, if 
the page on which the record is stored doesn’t have 
enough free space to accommodate the extra 14 bytes, 
space must be created—just as if a variable-length 
column was being updated to be longer. 

The free space is created by performing a page- 
split operation, which moves several records to 
a newly allocated page. This creates fragmenta- 
tion because the newly allocated page is almost 
never physically contiguous to the original page. 
Aside from creating fragmentation, page splits 
are very costly because all parts of the opera- 
tion must be fully logged. (For more informa- 
tion, see my blog post “How expensive are page 
splits in terms of transaction log?” at www 
-Sqlskills.com/BLOGS/PAUL/post/How-expensive- 
are-page-splits-in-terms-of-transaction-log.aspx.) 

You can alleviate the likelihood of page splits 
and fragmentation by using a non-zero fill factor 
(the default is to have each page 100 percent filled) 
to provision free space during index maintenance to 
allow for the extra 14 bytes for each versioned record 
during your update activity. 


Choosing a Good Index Fill 


Factor 


*ve been reading all of your recent blog posts 
i about how some index keys can lead to frag- 

mentation (and we have some that are GUIDs, 
unfortunately). Pd like to reduce the amount of 
fragmentation that occurs, but I can’t change the 
database schema at all. Should I set a default fill 
factor for the instance? 


This answer ties in nicely with the previous Q&A 
“The Cause of Fragmentation from Using Snapshot 
Isolation,” where one of the workarounds is to use an 
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index fill factor. You're right that making use of index 
fill factors is a great way to avoid fragmentation if 
you're unable to change the indexes themselves, but I 
would strongly advise against setting an instance-wide 
fill factor using sp_configure. 

As background for other readers, an index fill 
factor instructs the storage engine to leave some free 
space in leaf-level index pages during index build 
or rebuild operations. (Note that the fill factor isn’t 
maintained during regular insert, update, and delete 
operations.) 
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The default fill factor is 100, which instructs the 
storage engine to make the index leaf-level pages 100 
percent full (in other words leaving no space). This 
means that pages don’t have space to allow random 
record inserts (or records increasing in size), so page 
splits can occur, causing fragmentation. 

The tricky part about setting a fill factor is deter- 
mining which number to use for each index. If you set 
an instance-wide fill factor, there will likely be many 
indexes that don’t need a fill factor set, so they'll con- 
tain a lot of wasted, empty space. This can be espe- 
cially true of clustered indexes in which the clustering 
key and access pattern don’t cause fragmentation. 

For those indexes in which fragmentation is a prob- 
lem and your only recourse is to use a fill factor com- 
bined with regular index maintenance, you need to find 
a balance between how low to set the fill factor and how 
often you can rebuild the index to reset the free space in 
the index leaf-level pages. So what’s the secret? 

The unfortunate answer is that there isn’t a secret 
to which number to pick, but there’s a pretty simple 
methodology you can use to help you set an appropri- 
ate fill factor for each of your problem indexes. 


I usually recommend picking a fill factor value 
and putting it in production, and I often pick 70 
percent as the starting value. Monitor the frag- 
mentation over a period of a week or two and then 
decide whether to tweak the fill factor value up or 
down, or increase or decrease the frequency of index 
maintenance. 

You can monitor the fragmentation using the sys 
.dm_db_index_physical_stats DMV, which replaces 
DBCC SHOWCONTIG (that I wrote for SQL Server 
2000) in SQL Server 2005 and later. Just be care- 
ful how you run the DMV because it can be really 
expensive. (For an in-depth look at the DMV, see my 
blog post “Inside sys.dm_db_index_physical_stats” 
at www.sqlskills.com/BLOGS/PAUL/post/Inside- 
sysdm_db_index_physical_stats.aspx.) 

For some of my clients, Pve helped them set 
index fill factors as low as 50 percent because the 
extra space usage is preferable to the run-time cost of 
performing page splits. Your mileage will vary—just 
don’t fall into the trap of setting a default fill factor 
for the whole instance because this invariably leads to 
far too much wasted space. 


Is the Entire Clustering Key 
Duplicated in Nonclustered 


Indexes? 


s the entire clustering key duplicated in the 
nonclustered indexes? What if I have a wide 
clustering key? 


Every column of the clustering key must be included 
in every nonclustered index, but only once. And the 
order of the columns in the nonclustered indexes 
doesn’t have to match those of the clustered index. 
So if you have a wide key, you can make your non- 
clustered indexes unnecessarily wide. However, if a 
column that’s in the clustering key is already defined 
as a column of the nonclustered index, it’s not dupli- 
cated twice. For example, if you created the following 
clustered and nonclustered indexes: 


CREATE UNIQUE CLUSTERED INDEX CLInd ON 
T1 (col6, col4, cols) 


CREATE UNIQUE NONCLUSTERED INDEX NC1 ON 
T1 (col5, col4, col12) 


SQL Server will append only columns col6 and col8 
to the nonclustered index because col4 is already 
a member. And, because NCI is unique, these col- 
umns will need to be appended only in the leaf level 


of the nonclustered index. To be more specific, the 
leaf and the non-leaf levels (the B+ tree that’s used 
for navigation) can be different. SQL Server needs 
only what’s necessary for navigation in the tree. If the 
nonclustered index is unique, then only the columns 
defined by the nonclustered index must be in the tree. 
However, if NC1 had been non-unique, the clustering 
key would have needed to have been added to the tree 
for navigation. 

To give you a bit more insight, I’ll break this down 
into the columns needed for the leaf level and non- 
leaf levels of the nonclustered index for the following 
nonclustered examples: 


CREATE UNIQUE NONCLUSTERED INDEX NC1 ON 
T1 (col5, col4, col12) 


Non-leaf level(s): col5, col4, col12 
Leaf level: col5, col4, col12, col6, 
col8& 


Here, because the index is unique, SQL Server doesn’t 
need to add the clustered index columns to the tree. 
Navigation can occur easily with the unique key 
structure. 
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CREATE NONCLUSTERED INDEX NC2 ON T1 (col5, 
col4, col12) 


Non-leaf level(s): col5, col4, col12, 


col6, col8 
Leaf level: col5, col4, col12, col6, 
col8 


In this example, because the index is non-unique, 
navigation could be problematic because of signifi- 
cant duplicates in the nonclustered key. To eliminate 
any confusion, the clustered key columns are added to 
the tree. Because the clustered index is always unique 


Understanding 


hy doesn’t SQL Server use the RID for 
Wee from a nonclustered index into 

a clustered table? 
Actually, that’s almost how versions prior to SQL 
Server 7.0 worked—SQL Server used a volatile RID, 
which would follow the row on every update. More 
specifically, when a row was updated in these earlier 
versions and the row needed to physically move 
within the clustered index, all the nonclustered indexes 
needed to be updated as well. Given that you can have 
249 nonclustered indexes on each table, the work to 
update these nonclustered indexes could become ter- 
ribly expensive (in terms of time, disk space, logging, 
etc.). In SQL Server 7.0, the architecture was changed 
to use the clustering key (instead of a volatile RID) 
when the table is clustered. This is an optimization so 
that even when a row must be relocated, the nonclus- 
tered indexes don’t need to be updated (as long as the 
clustering key value doesn’t change). 

It’s important to note that there’s a similar 
optimization in heaps (i.e., tables that don’t have a 
clustered index) where they use a fixed/physical RID 
for lookups. Even when the record is relocated, this 
“fixed” RID doesn’t change. Instead—and only for 
heaps—SQL Server 7.0 uses a forwarding record to 
point from the original (i.e., fixed) physical RID to 
the new location of the record (known as a forwarding 
pointer). Even if the row moves again, the forwarding 
pointer will point only once—from the original RID 
to the new location. 

By only modifying one record (the forwarding 
pointer in a heap or the row itself in a clustered table) 
and not modifying all the index rows that point to 
it, SQL Server significantly reduced the overhead 
for nonclustered indexes. These structural changes 
let you have more nonclustered indexes without any 
additional overhead during data modifications, even 
when records physically move. I’m not suggesting that 
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(either explicitly or internally), all nonclustered 
indexes are actually unique as well (either explicitly 
or internally by adding the clustering key). 

I have created a version of sp_helpindex that 
shows the exact structures of your index. This pro- 
cedure is available in my blog post “A new and 
improved sp_helpindex (jokingly sp_helpindex8)” at 
www.sq|skills.com/BLOGS/KIMBERLY/post/A-new- 
and-improved-sp_helpindex-(jokingly-sp_helpindex8) 
.aspx. In addition to providing the information usually 
produced by sp_helpindex, it also produces the index_ 
id, the included columns, the filter, and finally the exact 
columns in the leaf level and the non-leaf levels. 


Index Lookups 


you create a lot of nonclustered indexes—other forms 
of overhead still exist, but this particular expense 
has been eliminated. These optimizations reduce 
the overall metadata modifications and logging, and 
therefore improve performance. This is another rea- 
son for choosing a specific clustering key. Choosing 
a clustering key value that’s heavily updated would 
be a very bad choice because it would cause a lot of 
record movement as well as require an update for each 
of your nonclustered indexes. 

Some people have also asked me why the SQL 
Server team chose to use the clustering key and not 
just use the RID approach for clustered tables. First 
and foremost, data can move a lot more in a clustered 
table as the table’s order has to be maintained. A fixed/ 
physical RID really wouldn’t work here. Second, and 
this is a bit of an optimization as well, the RID is an 
8-byte value that requires space to store it within the 
nonclustered indexes, and it’s not “real” data. If actual 
data could be used within nonclustered indexes, we 
might be able to save space and get more use out of our 
nonclustered indexes because they would automatically 
have an additional column of data in the index. And 
when the clustering key is narrow, you won't unnec- 
essarily waste space. However, this can go horribly 
wrong, too. What if you were to place the clustered 
index on a column such as LastName? SQL Server 
lets you create this clustering key; however, behind the 
scenes it will enforce uniqueness by adding an integer 
to the values that are non-unique. This value is added 
to the data row (at insert) and is automatically part of 
every nonclustered index. This value is necessary to 
make sure that every row is identifiable from the non- 
clustered indexes, but it wastes both time and space. 

So choose your clustered index wisely and choose 
it well. You want a value that’s naturally unique, nar- 
row, and static. Ideally, the value should also reduce 
fragmentation by not having too many insertion 
points. SQL 
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to recognize the hottest products among the past year’s offerings. Choosing favorites from such a com- 
petitive field—not to mention in the midst of such a financially unstable market—can be a challenge, but f 
this year’s winners show an uncommon breadth of functionality and originality. 
Our Editors’ Best program highlights products that SOL Server Magazine editors and contributors believe are 
worthy of recognition, whereas our Community Choice program lets our readers decide which products are the 
best. As we did last year—rather than presenting a predefined list of products and services that limited your selec- 
tion to our choices—we opened up the Community Choice nomination process to all. We let you nominate your 
favorite products and services, built the voting survey from there, and let everyone participate in the final voting 
phase. For the Gold winners of both Editors’ Best and Community Choice, we even reached out to readers and 
spoke to them about how the product or service helps them do their job. We spoke to real users about real experi- wi 
ences, and we hope these testimonials benefit you in your environment. — 
In these pages, you'll find our Gold, Silver, and Bronze Editors’ Best winners in each category 
directly adjacent to our Community Choice winners. Sometimes our editors and readers have 
agreed on favorite products and services in a given category, but more often they haven't. 
Do you agree with the choices our editors have made? Or do the picks that our read- 
ers have made carry more weight? Let us know! Regardless of whether these win- 
ners were chosen by editors or readers, you can be sure that all these products 
are worthy of serious consideration if you're in the market for a new tool. 


S QL Server Magazine’s annual Editors’ Best and Community Choice award programs offer a unique way 
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The polls have closed! 
Here are your—and our — 
favorite products of the year 


Best Backup and Recovery Software Product 


Editors’ Best 

Gold: LiteSpeed for SQL Server 6.0 ° 

Quest Software * www.quest.com 

Silver: SQL safe backup 6.4 * Idera * www.idera.com 
Bronze: Replay for SQL 4 * AppAssure ° 
www.appassure.com 


Why It Won: LiteSpeed for SQL Server 
sets a high bar for SQL Server backup 
products because it not only backs up SQL 
Server files quickly but also offers com- 
pression, encryption, and fast restores. 


Chris Burns, director and database consultant for 
Data Technology Consultants, said, “It works exact- 
ly the same as SQL Server, but with basically a more 
powerful engine. The high-speed, high-powered 
LiteSpeed engine backs up the way native backups 
would, but it’s honestly far faster and with compres- 
sion to boot.” He said the main benefit his team sees 
in LiteSpeed for SQL Server 6.0 is its SmartDiff 
backup technology. “We gain a substantial compres- 
sion ratio of about 75 to 80 percent of our physical 
standard SQL Server data files. The benefits of that 
aren't just in storage size—which relates to the cost 
per gigabyte—but also in restore speed, the obvious 
ability to restore more quickly, and also the ability 
to back up more quickly, which offloads the backup 
process from SQL Server as quickly as possible.” 


Community Choice 
Gold: SQL Backup Pro 6.4 * Red Gate Software ° 
www.red-gate.com 

Silver: SQL safe backup 6.4 ° 

Idera * www.idera.com 

Bronze: LiteSpeed for SQL Server 6.0 ° 
Quest Software * www.quest.com 


Other hot vote-getters in this year’s survey ... 

e Red Gate Software’s SQL HyperBac 

e OfficeRecovery.com’s Recovery for SQL 
Server 


“SQL Backup Pro 6.4 compresses and encrypts SQL 
Server backups, and it has extra features to strength- 
en backup activities, including network resilience for 
backups and an intuitive interface for powerful job 
management.” 


Best Business Intelligence and Reporting Product 


Editors’ Best 

Gold: Tableau Server 5.2 * Tableau Software ° 
www.tableausoftware.com 

Silver: NovaView 6.2 * Panorama Software ° 
www.panorama.com 

Bronze: Attunity SQL Server-CDC for SSIS ° 
Attunity * www.attunity.com 


Why It Won: Tableau Software’s business 
intelligence solution stands above the rest 
because it offers a rich, robust environ- 
ment that lets users share their data visu- 
alizations (e.g., reports, dashboards) with 
one another quickly and easily. 
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“We needed a product that was easy 
to implement, easy to use, flexible, 
very fast, robust, and reliable,” said 
Matt Krzysiak, COO for the Na- 
tional Motor Club. “Tableau met 
all of those criteria.” Krzysiak said 
Tableau Server lets his company’s 
business and IT personnel be more 
productive with data. “After instal- 
lation, we put Tableau and Tableau 
Server in the hands of our business 
people. They now have a tool that 
will allow them to visualize our data 
and publish visualizations on our intranet without 
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relying on our IT department. This has allowed us 
to move more quickly and discover information 
that has led to better products, better service, and 
reduced costs.” 


Community Choice 

Gold: Tableau Server 5.2 * Tableau 
Software * www.tableausoftware.com 
Silver: FastReport.Net * Fast Reports ° 
www.fastreports.com 


Bronze: Attunity SQL Server-CDC for SSIS + 
Attunity * www.attunity.com 


Other hot vote-getters in this year’s survey ... 
e SAP Business Objects’ Crystal Reports 
e SQL Sentry’s BI Suite 

e Panorama Software’s NovaView 6.2 


“Tableau Server just has the best embedded dash- 
board on the market.” 


e aa Mr] 
Best Database Management Product 


Editors’ Best 

Gold: SQL Toolbelt » Red Gate Software « 
www.red-gate.com 

Silver: SQL toolbox ° Idera * www.idera.com 
Bronze: Altiris Server Management Suite ° 
Symantec * www.symantec.com 


Why It Won: Red Gate Software’s SQL 
Toolbelt offers 14 tools in a popular set— 
they’re terrific tools and a great value. 


Steve Powell, the technical design lead at 
Spire Healthcare, is excited about using the tools 
in the SQL Toolbelt in SQL Server Management 
Studio (SSMS). One tool he especially likes is SQL 
Source Control, noting, “In a small shop like ours, 
Team Foundation Server (TFS) is a big ask. We use 
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free tools like SVN and TRAC for our code reposi- 
tory and bug tracking. Source Control costs peanuts 
compared with TFS, and it’s a much lighter tool to 
implement in terms of workflow and processes. We 
can use SQL Source Control to submit code changes 
directly from within SSMS. And attaching them to 
tickets couldn’t be simpler.” 


Community Choice 

Gold: SQL Toolbelt e Red Gate Software ° 
www.red-gate.com 

Silver: SQL admin toolset 1.5 ° Idera ° 
www.idera.com 

Bronze: SQL Toad for SQL Server * Quest Software ° 
www.quest.com 


Other hot vote-getters in this year’s survey ... 

e Embarcadero Technologies’ 
DBArtisan 

e Quest Software’s Spotlight on SQL 
Server Enterprise 

e SQL Sentry’s Event Manager 


“SQL Toolbelt really lives up to its name: It’s just a 
bunch of great tools.” 


Tech Product 
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DISCOVER WINDOWS IT PRO VIP 


Windows IT Pro VIP is the perfect tool for the IT pro who knows that 
15 minutes searching the Web is costing more than just time. 
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—over 41 000 articles on DVD and online, some exclusively for VIP members 
3. Broad—all the aE solutions, and FAQs ever published in: 
Windows IT Pro 
SQL Server Magazine 
SharePointPro Connections 
DevProConnections 
4. Reliable—every solution has been road-tested by our experts 
5. Impartial—with technical editors who are shaping the industry 


6. Economical—more than $1,000 of resources for less than $17* a month 


Upgrade to VIP at windowsitpro.com/go/vip 


* Rates vary outside the U.S. 
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Best Database Monitoring and Performance Product 


Editors’ Best 

Gold: Ignite 8 ° Confio Software * www.confio.com 
Silver: SQL diagnostic manager 6.2 SP1 + Idera ° 
www.idera.com 

Bronze: Spotlight on SQL Server Enterprise 7.0 ° 
Quest Software * www.quest.com 


Why It Won: Confio Software’s Ignite 8 is 

a true all-in-one performance-monitoring 

solution that builds off of Ignite’s original 

response-time monitoring functionality to 

also offer routine performance statistics 
via intuitive dashboards. 


Ben Miller, the director of database 
administration at Nature’s Way, said 
he was impressed by Ignite 8’s non-in- 
trusive nature and the speed at which 
information was gathered. “We had 
=- access to the application name, query 
text, executions, and reads and writes. 
It was more information in one place 
than we had from any other product 
we saw,” Miller said. “Everyone is familiar with 
slowness at times, but before [users] see the slowness 
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become common, we are in [Ignite] with alerts as 
well as rich graphs telling us which queries to worry 
about. We have actually tuned queries before users 
noticed, and they reciprocated with comments about 
things being faster before they had to complain.” 


Community Choice 

Gold: SQL diagnostic manager 6.2 SP1 • Idera ° 
www.idera.com 

Silver: SQLCentric * Pearl Knowledge Solutions ° 
www.pearlknows.com 

Bronze: SQL Response 1.3 * Red Gate Software ° 
www.red-gate.com 


Other hot vote-getters in this year’s survey ... 

e Quest Software’s Foglight Performance 
Analysis for SQL Server 

e SQL Sentry’s Performance Advisor for 
Analysis Services 

e Embarcadero Technologies’ DB Performance 
Center XE 


“SQL diagnostic manager is easy to use, very intui- 
tive, and helps me keep my production servers up all 
the time.” 


Editors’ Best 

Gold: Toad for SQL Server ° 
Quest Software ° 
www.quest.com 

Silver: PowerShell Plus 3.5 ° 
Idera * www.idera.com 
Bronze: CodeRush « 
DevExpress ° 
www.devexpress.com 


Trend 


Why It Won: 
Quest Software’s 
Toad for SQL 
Server offers an 
extremely wide 
range of functionality for 
developers and DBAs. 


TOAD 


Best Development Product 


“Quest Toad for SQL Server has been my editor of 
choice for the standard and enterprise SQL Server 
work I’ve performed since 2005,” said Hank Free- 
man, a senior systems database and data ware- 
house architect in Atlanta, Georgia. “It has al- 
ways been and continues to be the leading-edge 
tool for the professional DBA and developer.” 


Community Choice 

Gold: PowerShell Plus 3.5 + Idera * www.idera.com 
Silver: Toad for SQL Server * Quest Software ° 
www.quest.com 

Bronze: SQL Compare 8.2 * Red Gate Software ° 
www.red-gate.com 


Other hot vote-getters in this year’s survey ... 
e Red Gate Software’s SQL Prompt 4.0 
e DevExpress’s CodeRush 
e SoftTree Technologies’ SQL 
Assistant 5.1 


“PowerShell Plus features hundreds of free scripts, 
an advanced script editor and debugger, and a com- 
prehensive learning center, all integrated into a sin- 
gle product.” 
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Best Security/Auditing/Compliance Product 


Editors’ Best 

Gold: SQL compliance manager 3.2 « Idera ° 
www.idera.com 

Silver: Security Explorer 7.5 ° ScriptLogic ° 
www.scriptlogic.com 

Bronze: Hedgehog Enterprise 3.5 + Sentrigo ° 
www.sentrigo.com 


Why It Won: Today’s compliance regula- 
tions require many companies to be able 
to provide an audit trail that shows who 
is accessing data and when—a poten- 
tially time-consuming, complex process. 
Idera’s SQL compliance manager lets DBAs easily 
track user accounts and activity to prove to audi- 
tors what’s being done on SQL Server systems. 


Jon McLendon, a senior database administrator for 
Colonial Properties Trust, said that his company 
implemented this product to address his company’s 
SOX audit needs. “From an internal standpoint, it 
gave us the ability to very closely monitor and track 
the critical administration accounts for SQL Server.” 


idera 


TOOLS FOR SQL SERVER, 
SHAREPOINT AND POWERSHELL 


McLendon went on to say that the product lets his 
team know via email whether the SA account is 
successfully or unsuccessfully used. If it’s success- 
fully used, “any activity is tracked—we can go back 
and look over everything that was done under that 
login.” The product also tracks all other administra- 
tive accounts and their activity, said McLendon. “So 
there’s never any question about what we're doing or 
not doing in a system.” According to McLendon, 
his company has benefitted from SQL compliance 
manager because “it goes above and beyond what 
would be expected from the auditors.” 


Community Choice 

Gold: SQL compliance manager 3.2 « Idera ° 
www.idera.com 

Silver: Hedgehog Enterprise 3.5 ° 

Sentrigo * www.sentrigo.com 

Bronze: Security Explorer 7.5 ° 

ScriptLogic * www.scriptlogic.com 


Other hot vote-getters in this year’s survey ... 

e Idera’s SQL Secure 2.6 

e NetWrix’s SQL Server Change 
Reporter 


“SQL compliance manager is essential to ensuring a well 
secured and compliant SQL Server environment.” 


Best Storage Hardware Product 


Editors’ Best 

Gold: DroboElite * Data Robotics * www.drobo.com 
Silver: PowerVault * Dell * www.dell.com 

Bronze: HP StorageWorks * HP * www.hp.com 


Why It Won: The Drobo Elite is truly a 
set-it-and-forget-it iSCSI backup system. 
It’s inexpensive, easy to use, and yet pow- 
erful for the small-to-midsized business 
(SMB) storage environment. 


“The DroboElite fits well into my environment,” 
said Ivan Breit, lead systems administrator for 
Whalen Furniture. “My previous storage system 
had failed, and I replaced it with the DroboElite, 
which ended up saving me thousands of dollars 


ba 


x 
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compared with other systems 
I was considering. Since imple- 
menting the DroboElite, my 
VMware backup environment 
has performed flawlessly. I’m 
also considering doing some 
restructuring of my virtualized 
environment and I am going to 
use the iSCSI connectivity of 
the DroboElite to provide primary storage for my 
VMs during the transition. The DroboElite is a flex- 
ible storage solution that meets my needs, and the 
BeyondRAID technology does exactly what it’s sup- 
posed to do.” 


Gold: EMC CLARiiON series * EMC ° 
www.emc.com 

Silver: Dell PowerVault series * Dell * www.dell.com 
Bronze: NetApp FAS3100 series ° 

NetApp ° www.netapp.com 


Other hot vote-getters in this year’s survey ... 
e Dell’s EqualLogic series 
í e IBM’s XIV Storage System 
x e HP StorageWorks solutions 
“EMC CLARION provides terrific bang for the 
buck.” 


Best Free Tool 


Editors’ Best 

Gold: SP_WholsActive * Adam Machanic * 
sqlblog.com 

Silver: SSMS Tools Pack 1.9 » Mladen Prajdic * 
www.ssmstoolspack.com 


Most Overused 


Cloud 


Business intelligence 


Data BUZZwords 
Seamless 


Robust 


Dynamic Low-hanging fruit 
Real-time Set-based 
Heterogeneous One true view 


Favorite Consumer 


Fech Product 


Apple iPhone Apple iPad 
Google Android GPS devices 
Microsoft Xbox 360 Digital meat thermometer 


Apple iPod Nano 


Microsoft Zune 
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USB drink chiller 


VCR 


Bronze: SQL Server Maintenance Solution ¢ 
Ola Hallengren ° ola.hallengren.com 


Why It Won: SP_WholsActive is men- 

* tioned in the SQL Server community as 

gy the epitome of a fantastic free tool. Adam 

papei sa Machanic updates the tool frequently, it’s 

. ™ well explained and supported by great in- 

ternal documentation, and its code is considered to 
be the best of the best. 


“To say I like SP_WholsActive is an understate- 
ment,” SQL Server Magazine author Kevin Kline 
said in “SP_WholIsActive” (June 2010, InstantDoc 
ID 125107). “This is probably the most useful 
and effective stored procedure I’ve ever encoun- 
tered for activity monitoring. The purpose of the 
SP_WholsActive stored procedure is to give DBAs 
and developers as much performance and work- 
load data about SQL Server’s internal workings 
as possible, while retaining both flexibility and 
security.” 


Gold: SQL Server Maintenance Solution ° 
Ola Hallengren ° ola.hallengren.com 
Silver: SQL check 2.6 ° Idera * www.idera 
.com 

Bronze: Tableau Public * Tableau Software ° 
www.tableausoftware.com 


Other hot vote-getters in this year’s survey ... 
N F o Quest Software’s Toad Data Modeler 
D Freeware Edition 
parai ` Red Gate Software’s SQLSearch 1.0 

™ e LessThanDot’s SQLCop 


“Ola Hallengren’s tools are justly famous.” SQL] 
InstantDoc ID 128822 
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SQL Server Magazine Congratulates Ider 


SQL Server Magazine — Community Choice Awards — Gold 


soi) Category: Best Database Monitoring/Performance Product 
Product: Idera SQL Diagnostic Manager 


Category: Best Development Tool 
Product: Idera PowerShell Plus with SQL scripts 


Category: Best Security/Auditing/Compliance Product 
Product: Idera SQL compliance manager 


Category: Best Vendor Tech Support 


Category: Best Backup/Recovery Product 
Product: Idera SQL safe 


Category: Best Database Management Product 
Product: Idera SQL admin toolset 


Category: Best Free Tool 
Product: Idera SQL check 


Category: Best Security/Auditing/Compliance Product 
Product: SQL compliance manager 3.2 


Category: Best Backup/Recovery Product 
Product: SQL safe backup 6.4 


Category: Best Database Management Product 
Product: SQL toolbox 


Category: Best Database Monitoring/Performance Product 
Product: SQL diagnostic manager 6.2 SP1 


Category: Best Development Tool 
Product: PowerShell Plus 3.5 


Learn more about idera products at www.idera.com 


Left-Brain.com is the online superstore stocked with 
educational, training, and career-development materials 
focused on meeting the needs of IT professionals like you. 


Featured Product: 
T-SQL 101: A Quick-Start Guide to T-SQL Basics 


-SQL 1 01 4 The 10 lessons in this eBook will help you learn the basics, such 
A Quick-Start as how to write queries that retrieve, add, update, and delete 
Guide to records in tables and how to group and aggregate data that 
T-SQL Basics you retrieved with queries so that you can produce informative 


reports. By the end, you'll even be able to create tables, views, 
and custom stored procedures. 


Download your copy today for only $15.95*! 
sqimag.com/go/left-brain/tsq| 


www.left-brain.com 


Row Value 


Constructor 


This feature makes code less verbose, more 
readable, and easier to maintain 


T he ISO SQL Standard defines a feature called 
row value constructor, which in essence is a 
value or a list of values that construct a row. 
This feature is also referred to as vector expression. An 
example of a row value constructor is 

ROW(191, '28100212', 10.10) 

The use of the ROW keyword is optional. In database 
platforms that support this feature, developers often 
omit it, like so: 

(181, '28100212', 10.10) 

The ordinal positions of the elements in this construct 
are significant, as they are in other SQL column lists 
(e.g., SELECT, ORDER BY). Hence this construct 
can’t be considered a true equivalent of a tuple from 
the relational model. Some relational model purists 
advise against its use for this reason. I find this feature 
useful; it makes the code less verbose, more readable, 
and easier to maintain. 

Standard SQL supports the use of row value 
constructors in many language elements. SQL Server 
2008 introduced very minimal support for the feature 
in the form of an enhanced VALUES clause that 
allows defining multiple rows. The enhanced VAL- 
UES clause can be used in an INSERT statement and 
to define a derived table. (For details, see “SQL Server 
2008’s T-SQL Features,” InstantDoc ID 101153.) But 
a lot of functionality related to row value constructors 
is still missing as of SQL Server 2008 (including R2). 
In this article I introduce the missing functionality. If 
you think this functionality is important, you can vote 
for Hugo Kornelis’s request to add the feature to SQL 
Server. Go to the Microsoft Connect page at connect 
-microsoft.com/SQLServer/feedback/details/299231. 

Most of the code that I present in this article won’t 
run on SQL Server because the features aren’t yet 
supported. However, you can run logically equivalent 
code that’s currently supported; use Listing 1 to create 
the sample data used by those code samples. 
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Some of the samples refer to an argument called 
@key, as well as arguments called @cl, @c2, and 
@c3. In those examples you can use the following 
code to declare and initialize those variables: 


DECLARE @key AS INT = 2; 

DECLARE 
@p1 AS INT = 201, 
@p2 AS DATETIME = '2100212', 
@p3 AS NUMERIC(12, 2) = 10.10; 


Itzik Ben-Gan 


(Itzik@ SolidQ.com) is a mentor with Solid 
Quality Mentors. He teaches, lectures, and 
consults internationally. He’s a SQL Server MVP 
and is the author of several books about 
TSQL induding Microsoft SQL Server 2008: 
FSQL Fundamentals (Microsoft Press). 


Assignment 

The SQL standard supports assignments in an 
UPDATE statement using row value constructors, 
like so (don’t try to run this code, because it’s not 
supported as of SQL Server 2008 R2): 


ORE on the WEB 


Download the listings at 
InstantDoc ID 128810. 


UPDATE dbo.T1 
SET (cl, c2, c3) = (@pl, @p2, 
@p3) 
WHERE keycol = @key; 


The logically equivalent form supported in SQL 
Server is 


UPDATE dbo.T1 


SET cl = @pl, 
c2 = @p2, 
c3 = @p3 


WHERE keycol = @key; 


This example isn’t very exciting and doesn’t clearly 
demonstrate the advantage of row value constructors; 
the true benefit becomes apparent when used with a 
subquery: 


UPDATE dbo.T1 
SET (cl, c2, c3) = (SELECT T2.cl1, 
T2.c2, T2.c3 
FROM T2 
WHERE T2.keycol = T1.keycol) 
WHERE keycol = @key; 
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pe 


LISTING |: Code to Create Sample Data 


SET NOCOUNT ON; 
USE tempdb; 


IF OBJECT_IDC'dbo.T1', 'U') IS NOT NULL DROP TABLE dbo.T1; 
GO 


CREATE TABLE dbo.T1 
¢ 


keycol INT NOT NULL, 
grpcol VARCHAR(19) NOT NULL, 
tel INT NOT NULL, 
c2 DATETIME NOT NULL, 
c3 NUMERIC(12, 2) NOT NULL, 


CONSTRAINT PK_T1 PRIMARY KEY(keycol), 
CONSTRAINT UNQ T1 cl c2_c3 UNIQUE(cl, c2, c3) 
Ja 


INSERT INTO dbo.T1l(keycol, grpcol, cl, c2, c3) VALUES 
( 2, 'A', 101, A 19.19), 
C3, 'A', 101, '20190212', 10.20), 
C5, 'A', 101, '20100213', 10.10), 
C7, 'A', 102, '20190212', 19.19), 
(11, 'A', 102, '20100212', 18.29), 
(13, 'B', 201, '20100212', 10.19), 
Glz BL Aa AAA 20) ie 
(19, 'B', 202, '20100212', 18.19), 
(23, 'B', 202, '20100213', 18.19), 
(29, 'B', 202, '20100213', 18.29); 

GO 


IF OBJECT_IDC'dbo.T2', 'U') IS NOT NULL DROP TABLE dbo.T2; 
GO 


CREATE TABLE dbo.T2 


¢ 
keycol INT NOT NULL, 
cl INT NOT NULL, 
c2 DATETIME NOT NULL, 
c3 NUMERIC(12, 2) NOT NULL, 


CONSTRAINT PK_T2 PRIMARY KEY(keycol), 
CONSTRAINT UNQ T2_ cl c2_c3 UNIQUE(c1, c2, c3) 
DE 


INSERT INTO dbo.T2(keycol, c1, c2, c3) VALUES 
C 2, 301, '2@110212', 20.10), 
E 3, 301, '2@110212', 20.20), 
C5, 301, '2@110213', 20.10), 
C7, 302, '2@110212', 20.10), 
(11, 302, '20119212', 20.20), 
(43, 4@1, '20110212', 20.10), 
(17, 4@1, '20110212', 20.20), 
(19, 402, '20110212', 20.10), 
(23, 4@2, '20110213', 20.10), 
(29, 482, '20110213', 20.20); 


This is logically equivalent to 


UPDATE dbo.T1 
SET c1 = (SELECT T2.c1 
FROM T2 
WHERE T2.keycol = Tl.keycol), 
c2 = (SELECT T2.c2 
FROM T2 
WHERE T2.keycol = T1.keycol), 
c3 = (SELECT T2.c3 
FROM T2 
WHERE T2.keycol = T1.keycol) 
WHERE keycol = @key; 


The form that uses a row value constructor is less 
verbose and lends itself to better optimization. 
Figure 1 shows the plan I got for the form that’s 
currently supported by SQL Server (with multiple 
subqueries). Note that the plan shows a separate visit 


to the row in T2 for each subquery. 
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There are existing supported alternatives in SQL 
Server that do optimize well, including the non- 
standard join-based UPDATE statement and the 
standard MERGE statement. But the suggested 
UPDATE syntax that uses row value constructors, in 
addition to being standard, is also clearer and more 
elegant than the supported alternatives. 


Comparison Predicates 

Standard SQL also supports comparison predicates 
that use row value constructors. The general form of 
such a predicate is 


<RVC1> <comparison_operator> <RVC2> 

The operators equals (=) and not equal to (<>) follow 
the rules for equality operations in the standard. An 
example for use of the equals operator is 

SELECT keycol, cl, c2, c3 

FROM dbo.T1 

WHERE (c1, c2, c3) = (@p1, @p2, @p3); 


This is logically equivalent to 


SELECT keycol, cl, c2, c3 


FROM dbo.T1 

WHERE cl = @p1 
AND c2 = @p2 
AND c3 = @p3; 


Regarding treatment of NULLs, remember that SQL 
follows three-valued predicate logic, meaning that a 
comparison of two NULLs or a non-NULL value 
with a NULL yields UNKNOWN. In predicate- 
based query filters such as WHERE, UNKNOWN 
is filtered out. Standard SQL supports an interesting 
predicate called distinct predicate that has the syntax 
IS [NOT] DISTINCT FROM. It can be used in two 
ways—IS NOT DISTINCT FROM yields TRUE 
when both sides are known to be equal or when both 
are NULLs, and FALSE otherwise. IS DISTINCT 
FROM yields TRUE when the two sides are known to 
be different or when one side is NULL and the other 
isn’t, and FALSE otherwise. SQL Server currently 
doesn’t support this predicate (as of SQL Server 
2008 R2), but if you find it useful, you can vote for 
Steve Kass’s recommendation to add it. Go to the 
Microsoft Connect page at connect.microsoft.com/ 
SQLServer/feedback/details/286422. 

In standard SQL you can compare two row value 
constructors and consider two NULLs as equal for 
this purpose (or more accurately, as not being distinct 
from each other), like so: 


WHERE (c1, c2, c3) IS NOT DISTINCT FROM 
(@p1, @p2, @p3); 
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tel te] te] ay 
Nested Loops + Nested Loops . Nested Loops i Clustered Index Seek (Clustered) 
(Left Outer Join) oppass s ai (Left Outer Join) oa Se (Left Outer Join) [T1]. [PK_T1] 
Cost: 0 % n Cost: 0 % . Cost: 0 % Cost: 10 % 
ty 
Clustered Index Seek (Clustered) 
[T2]. [PK_T2] 
Cost: 10 % 
Clustered Index Seek (Clustered) 
[T2]. [PK_T2] 
Cost: 10 % 
L Clustered Index Seek (Clustered) 
[T2] . [PK_T2] 
Figure | 


Plan for UPDATE statement 


The equals operator for row value constructor can be 
very handy when expressing a composite join: 


SELECT T1.keycol AS T1_key, T2.keycol AS 
T2_key 
FROM dbo.T1 JOIN dbo.T2 
ON (T1.c1, T1.c2, T1.c3) = (12.c1, 
T2.c2, T2.c3); 


Similarly, the SQL standard supports the not equal to 
operator with vector expressions: 


SELECT keycol, c1, c2, c3 
FROM dbo.T1 
WHERE (c1, c2, c3) <> (@p1, @p2, @p3); 


This is logically equivalent to the currently supported 
form: 


SELECT keycol, c1, c2, c3 
FROM dbo.T1 
WHERE cl <> @p1 

OR c2 <> @p2 

OR c3 <> @p3; 


And again, in standard SQL you can compare two 
row value constructors and consider a non-NULL 
value versus a NULL as being distinct from each 
other, like so: 


WHERE (cl, c2, c3) IS DISTINCT FROM (@p1, 
@p2, @p3); 


The operators less than (<), greater than (>), less than 
or equal to (<=), and greater than or equal to (>=) 
follow the rules for ordering operations in the stan- 
dard. Use of these operators with vector expressions 
can significantly reduce the length and complexity of 
predicates, as well as lends itself to good optimization. 
Following are examples of predicates and their logical 
equivalents that SQL Server does support (as of SQL 
Server 2008 R2). 
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Less than operator: 


SELECT keycol, c1, c2, c3 
FROM dbo.T1 
WHERE (c1, c2, c3) < (@p1, @p2, @p3); 


Logically equivalent to: 

SELECT keycol, c1, c2, c3 

FROM dbo.T1 

WHERE (c1 < @p1) 

OR (c1 = @p1 AND c2 < @p2) 

OR (c1 = @p1 AND c2 = @p2 AND c3 < @p3); 
Greater than operator: 

SELECT keycol, c1, c2, c3 

FROM dbo.T1 

WHERE (c1, c2, c3) > (@p1, @p2, @p3); 


Logically equivalent to: 


SELECT keycol, c1, c2, c3 


FROM dbo.T1 
WHERE (c1 > @p1) 
OR (c1 = @p1 AND c2 > @p2) 


Il 


OR (c1 = @p1 AND c2 = @p2 AND c3 > @p3); 


Less than or equal to operator: 


SELECT keycol, c1, c2, c3 
FROM dbo.T1 
WHERE (c1, c2, c3) <= (@p1, @p2, @p3); 


Logically equivalent to: 


SELECT keycol, c1, c2, c3 

FROM dbo.T1 

WHERE (c1 = @p1 AND c2 = @p2 AND c3 <= 
@p3) 

OR (c1 = @p1 AND c2 < @p2) 

OR (c1 < @p1); 


December 2010 


27 


28 December 2010 


ROW VALUE CONSTRUCTOR 


Greater than or equal to operator: 


SELECT keycol, c1, c2, c3 
FROM dbo.T1 
WHERE (c1, c2, c3) >= (@p1l1, @p2, @p3); 


Logically equivalent to: 


SELECT keycol, c1, c2, c3 

FROM dbo.T1 

WHERE (c1 = @p1 AND c2 = @p2 AND c3 >= 
@p3) 

OR (c1 = @p1 AND c2 > @p2) 

OR (c1 > @p1); 


You can clearly see the benefits in the support for 
row value constructors by these operators in terms of 
reducing the length of code and complexity compared 
with the logically equivalent options. The benefits of 
the equals and not equal to operators are easy to see 
in business cases. As for the other operators, such as 
greater than, an example for a business case is paging. 
Suppose that you're implementing a paging solution 
that involves multiple sort columns (e.g., cl, c2, c3). 
For the paging solution to be deterministic, you need 
to ensure that together, the ordering elements uniquely 
identify a row. If not, you can add the primary key as 
a tiebreaker. At any rate, whenever the user asks for 
the next page, the application invokes a procedure, 
passing the values of the ordering elements from the 
last row in the last page retrieved. The procedure can 
then invoke a query that filters all rows greater than 
the passed anchor row, like so: 


SELECT TOP(@pagesize) keycol, cl, c2, c3 
FROM dbo.T1 

WHERE (c1, c2, c3) > (@p1, @p2, @p3) 
ORDER BY c1, c2, c3; 


This code is clear and concise, and it lends itself to 
good optimization provided an index is defined on 
(cl, c2, c3). 

It’s important to note that row value constructors 
don’t address a couple of needs related to this paging 
idea. First, the ordering direction of all elements is the 
same (ascending when using > and descending when 
using <). Second, treatment of three-valued logic 
in terms of NULLs is different than with ordering 
in the sense that a comparison with a NULL yields 
UNKNOWN and the row is filtered out. But this 
feature could be very useful as long as all ordering 
elements are defined as NOT NULL and you do need 
them all to be in the same direction. 


IN and BETWEEN 
After considering the operators equals, not equal to, 
less than or equal to, and greater than or equal to, it’s 


easy to see that row value constructors can also be 
used with IN and BETWEEN predicates. Here’s an 
example using IN: 


SELECT keycol, c1, c2, c3 


FROM dbo.T1 
WHERE (c1, c2, c3) IN (SELECT cl, c2, c3 
FROM T2); 


Assuming the subquery returns the rows R1, R2, ..., 
Rn, this query is then equivalent to 


SELECT keycol, c1, c2, c3 

FROM dbo.T1 

WHERE (c1, c2, c3) = R1 
OR (cl, c2, c3) = R2 
OR... 
OR (cl, c2, c3) = Rn; 


Here’s an example with BETWEEN: 


SELECT keycol, c1, c2, c3 
FROM dbo.T1 
WHERE (c1, c2, c3) BETWEEN (@p11, @p12, 
@p13) 
AND (@pr1, @pr2, @pr3) ; 


This is logically equivalent to 


SELECT keycol, cl, c2, c3 

FROM dbo.T1 

WHERE (c1, c2, c3) >= (@pl1, @p12, @p13) 
AND (c1, c2, c3) <= (@pr1, @pr2, @pr3) ; 


MIN and MAX Aggregates 

Earlier I showed how the SQL standard supports 
comparison predicates with row value constructors. 
The MIN and MAX aggregates also perform com- 
parisons, only across rows. So the idea of row value 
constructor comparisons can therefore be extended to 
aggregates as well. Here’s an example using MAX: 


SELECT grpcol, MAX(c1, c2, c3) 
FROM dbo.T1 
GROUP BY grp; 


Note that the goal here is to return the maximum c1 
value in each group, the maximum c2 value out of the 
rows with the maximum c1 value, and the maximum c3 
value out of the rows with the maximum c1 and maxi- 
mum c2 values. This doesn’t mean return the maximum 
of cl, c2, and c3. Some platforms support functions 
called GREATEST and LEAST that return the maxi- 
mum and minimum, respectively, out of the given list of 
values in the same row. Although it would be nice for a 
future version of SQL Server to include such support, 
that’s not the focus of this discussion. Logically, the 
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proposed MAX of a row value constructor is similar to 
what you can currently achieve in SQL Server, like so: 


WITH C AS 
C 
SELECT grpcol, cl, c2, c3, 
ROW_NUMBER() OVERCPARTITION BY grpcol 
ORDER BY c1 DESC, c2 DESC, c3 DESC) 
AS rownum 
FROM dbo.T1 
) 
SELECT grpcol, cl, c2, c3 
FROM C 
WHERE rownum = 1; 


One of the ways to avoid the confusion with the 
logic of the GREATEST and LEAST functions is to 
explicitly use the ROW keyword, like so: 


SELECT grpcol, MAX(ROW(cl, c2, c3)) 
FROM dbo.T1 
GROUP BY grpcol; 


Adding such support in a future version of SQL Server 
would be valuable. Such an aggregate could help solve 
business problems such as the TOP N Per Group prob- 
lem. For details about this problem, including a perfor- 
mance discussion about currently available solutions, 
see “Optimizing TOP N Per Group Queries,” 
InstantDoc ID 103663. In “Optimizing TOP N Per 
Group Queries” I cover three possible solutions: 

1. A solution based on APPLY that I 
recommend using with dense partitions when a 
good index is available. 

2. A solution based on ROW_NUMBER 
that I recommend using with nondense parti- 
tions when a good index is available. 

3. A solution based on a carry-along- 
sort concept (concatenation technique) that I 
recommend using regardless of density when 
there’s no good index in place and you can’t or 
don’t want to create one. 


Listing 2 contains the query I used to implement 
the third approach. With support for row value 
constructors in MIN and MAX aggregates, you 
can implement a similar idea only with far clearer 
code, like so: 


SELECT custid, MAX(orderdate, orderid, 
filler) 

FROM dbo.Orders 

GROUP BY custid; 


The benefit of such support for row value con- 
structors becomes clear when you compare this 
query with the one in Listing 2. 
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LISTING 2: Carry-Along-Sort Solution to Top N 
Per Group Problem 


WITH C AS 


SELECT. 
custid, 
MAX( (CONVERT (CHAR(8), orderdate, 112) 
+ RIGHT('ØØØØØØØØØ' + CAST(orderid AS VARCHAR(19)), 19) 
+ filler) COLLATE Latin1l_General_BIN2) AS string 
FROM dbo.Orders 
GROUP BY custid 
) 
SELECT custid, 
CAST(SUBSTRING(string, 1, 8) AS DATETIME ) AS orderdate, 
CAST(SUBSTRING(string, 9, 18) AS INT ) AS orderid, 
CAST(SUBSTRING(string, 19, 200) AS CHAR(2@0)) AS filler 
FROM C; 


Cast Your Vote 
In this article I introduced the concept of row value 
constructors from standard SQL and explained the 
different language constructs that can benefit from 
using it. This construct would add a lot of value if 
a future version of SQL Server included support for 
it. To vote for Hugo Kornelis’s request to add the 
feature to SQL Server, go to the Microsoft Connect 
page at connect.microsoft.com/SQLServer/feedback/ 
details/299231. SQL 
InstantDoc ID 128810 
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The Smart DBA's Guide to 


SOL Server 
Disaster Recovery, Part 2 


The final two Ds of our 7D Method" take 
you through the home stretch 


ou need a disaster-recovery program spe- 
cific to your SQL Server assets—a program 
developed and maintained by SQL Server 
professionals. You're not necessarily looking for a gen- 
eral high-availability plan; that’s a topic for another 
time. No, you want a data-centric disaster-recovery 
program that concentrates on SQL Server availability, 
backup, and continuity. And this plan needs to have a 
laser focus on baking resiliency into your systems so 
that, in the event of a local or regional disaster, your 
organization can continue its operations. 
We're applying our 7D Method to developing 
a SQL Server disaster-recovery process. The 7D 
Method is a decide and execute process comprising 
seven stages, although we're using only the first five 
Ds for this discussion. In “The Smart DBA’s Guide 
to SQL Server Disaster Recovery, Part 1” (InstantDoc 
ID 126089), we covered Discover and Design. In this 
article, we'll continue through Develop, Deploy, and 
the Day-to-Day management of the SQL Server 
disaster-recovery program you create. 


Get Robust! 
In Part 1, we spoke of the relative frequency and impact 
of four classes of interruption. Traditional disaster- 
recovery planning focuses on a “smoking pile of rubble” 
scenario, which assumes that by preparing for the worst, 
all lesser SQL Server interruption incidents will be 
adequately addressed. We challenge that assumption! 
The 7D Method can help you outline a robust 
program—one that includes disaster-recovery plan- 
ning, testing, analysis, and continuous improvement. 
In the end, you'll have a broad understanding of SQL 
Server disaster recovery, not just a planning and docu- 
mentation process. While you develop your program, 
we suggest dividing it into two perspectives: one for 
emergencies that require the servers to be in produc- 
tion offsite (i.e., for serious interruptions) and one 
for disruptions that don’t require leaving the building 
(.e., for less significant interruptions). 
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Each of these perspectives has different triggers 
for declaring an emergency, executing processes 
defined by your disaster-recovery plan, and begin- 
ning the recovery processes it specifies. Adopting two 
perspectives means applying new rules, deploying 
new resources, and determining whether external 
resources are necessary or whether you can recover 
using the resources you have on hand. 


The Third D: Develop 

Optimal preparedness from either of the aforemen- 
tioned perspectives requires that you measure and 
report using two types of key performance indicators 
(KPIs): Outcome Indicators (things that confirm 
that your program worked) and Activity Indicators 
(day-to-day protection practices or “in process” met- 
rics). For more information about KPIs, see the web 
sidebar, “KPIs, Outcome Indicators, and Activity 
Indicators” (InstantDoc ID 128919). With a shift in 
perception, your program will benefit if you borrow 
from the world of Maintenance, Repair, and Over- 
haul (MRO), whose focus is reliability. The MRO 
world analyzes incidents as chronic or sporadic. 

Chronic incidents. Chronic incidents are variations 
from an accepted range of performance. The desired 
outcome is to bring variance back to accepted toler- 
ances. Chronic incidents often fall into an “accepted 
losses” category—they’re negative deviations from 
performance norms but aren’t emergencies in and of 
themselves. They might fall under a “planned down- 
time” category, in which systems need to be taken 
offline for patches, updates, hardware refreshes, and 
so on. They’re often accepted as part of the job, such 
as warning messages from a failing disk in a RAID 
array or warnings that you're running out of space in 
the SQL Server transaction log. 

Sporadic incidents. Sporadic incidents are typi- 
cally more severe, and they're more infrequent and 
unpredictable, not to mention more costly. Sporadic 
incidents demand urgent attention. They’re time- 
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consuming to resolve and almost always carry a high 
dollar loss or human cost. We strongly encourage you 
to include near misses as well as actual incidents in 
your reviews of high-impact incidents because these 
occurrences are generally few and far between. You'll 
want as much testing of your preparation for such 
events as you can get so that when the real event occurs, 
everyone has already practiced their roles. 

Both chronic and sporadic incidents can be step- 
ping stones whose resolution requires higher levels of 
performance. Getting back to normal is adequate, but 
it isn’t progress. As we're defining things, normal is the 
status quo—but it’s not continuous improvement. 

Let’s get started with activities that you can do 
within the next 30 days to jump-start your SQL Server 
disaster-recovery program. When you finish, you'll be 
done with the Develop stage. 

Figure | shows a SQL Server disaster-recovery 
program framework. It’s a concept drawing that lays 
out the core processes and key activities of your pro- 
gram. The six columns represent the core processes; 
under each process, you should list the key activities 
necessary to successfully execute the process. These 
activities can be performed in parallel. For example, 
under Prepare, you can be budgeting at the same 
time that you're forming your disaster-recovery com- 
mittee. You'll need two of these visual models—one 


activity ti 


SQL/DR PROGRAM FRAMEWORK 


ON-SITE PERSPECTIVE 


SIX CORE PROCESSES 


PREPARE| |PREVENT| [RESPOND RECOVER | [INVESTIGATE] [REPORT 


Differing numbers of Key 
Activities are associated with 
each Core Process; Relevant 

Tasks (not shown) are associated 
with each Key Activity 


Figure | 


The SQL Server disaster-recovery program framework 
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for onsite-level incidents and one for offsite-level 
incidents. Try to limit the number of key activities to 
10; otherwise, the model will be too complex to be 
useful as a tool. Each key activity will have one or 
more associated tasks (not shown). 

Adopting this visual tool accomplishes several 
things. First, you get an alignment tool for busi- 
ness sponsors and the IT team, a common language 
and meaning that show a general view of desired 
outcomes, deliverables, and performance directives. 
Second, you get an instant answer to the question 
“Where are we?”—which eliminates long report writ- 
ing. We suggest coloring the activity/owner blocks 
red, yellow, or green to show progress (and quality, if 
you have best practices defined). 

Assign ownership of each activity to an indi- 
vidual. This person will handle the activity and the 
tasks required to perform it, including information 
requests from other activity owners. Each person will 
interact directly with peers, each of whom will be han- 
dling other key activities, thus eliminating the usual 
decision-making delay, waiting for marching orders 
before work gets done. This is a peer-oriented model 
that’s well suited to the informational requirements of 
a world-class SQL Server disaster-recovery program. 

A word of advice: Spend 70 percent of your 
time with the framework’s onsite perspective, and 30 
percent of your time working the offsite perspective, 
since most operational interruptions will be local. 
Your offsite perspective should limit itself to recov- 
ering your SQL Server environment to an offsite 
location; make sure it integrates with your enterprise 
disaster-recovery initiative, as well as your offsite data 
center recovery plan (if your organization has one). 
For more information about developing a plan, see 
the web sidebar “Avoid Traditional Project Manage- 
ment for SQL Server Disaster-Recovery Programs” 
(nstantDoc ID 128920). You're probably already 
doing many of the tasks that are part of a robust SQL 
Server disaster-recovery program (e.g., backups, log 
shipping, database mirroring, clustering, using third- 
party solutions). But have you looked at those tasks 
through the lens of disaster recovery? What questions 
do you have about these tasks? What don’t you like 
about what you're currently doing? What assumptions 
have you made about these operations, about your 
databases, and about your ability to prepare, prevent, 
respond, recover, investigate, and report? 

Which SQL Server competencies do you need in 
order to implement your SQL Server disaster-recovery 
program? What expertise, skill, and training must you 
and your resources obtain and maintain? Which sys- 
tem reports must you know how to run and interpret? 
What SQL Server functions must you know how to 
execute? What SQL Server decisions will you need to 
make during each incident type? What SQL Server 
administration traps can you predict and avoid? These 
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are the kinds of questions that you should be asking 
yourself as you populate the SQL Server disaster- 
recovery framework. 
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The Fourth D: Deploy 

The Deploy phase consists of performing six core 

processes in pairs, as Figure 2 shows: 

e Prepare & Prevent—These processes include 
documenting, testing, and training. You'll docu- 
ment roles, people, and processes; test responses 
to incidents and near misses; train in incident 
response; and evaluate that training. 

e Respond & Recover—These processes equate to 
the coordination and facilitation of people and 
processes during an incident or near miss. 

e Investigate & Report—These processes mean 
performing root cause and failure analysis. Figure 2 
You'll analyze information-sharing processes, do 
“lessons learned” exercises, analyze performance 
and execute follow-up, and manage change as it perform the Deploy phase’s six core processes, their 
becomes necessary. activities, and their associated tasks. As you can see 

in Figure 2, the lessons you learn from incidents and 

In Part 1 of this series, we covered the Discover near misses feed into benchmarks (best practices) 
and Design phases of the 7D Method, so by now and leading indicators (in-process metrics); these 
your program should have at least placeholders—and lessons raise the bar on benchmarks and strengthen 
hopefully extensive documentation—for how you the leading indicators. Therefore, the day-to-day best 
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BASELINE CHECKLIST 


Oo Backups done and tested periodically 
O Critical database and apps restore sequence documented 
O Scheduled jobs, DTS/SSIS packages, etc., scripted out and safely stored 


O Associated Windows jobs (FTP, Windows scheduled jobs) documented 
and safely stored 


O Current copy of passwords for SQL Server logins and encryption keys 


stashed in a safe place 


oO Windows backups of relevant log data (SQL Server Engine, Change Data, 


etc.) made and secure 


o Disaster-recovery program budgeted for (beyond backup) as a persistent 


line item 


o Disaster-recovery facility—cold site, warm site, hot site—configured and 


functional 


O Standby or failover servers in place and operational 
O Standby or failover servers continuously tested 
O Required firewall and perimeter security in place at disaster-recovery site 


O Documented procedure to transfer DNS settings during failover readily 


available 


O “Remote into disaster-recovery site” plan in place for SQL People group 


O SQL People primary computer for managing SQL Server systems is 


portable and can travel with SQL Server person; it’s set up to access 
both the primary site of operations and the disaster-recovery facility. 


practices for managing and defending your SQL 
Server installation become more robust and more 
comprehensive. Over time, this methodology should 
help you control and reduce the number of chronic 
incidents in your environment and enable you to 
better cope with sporadic incidents. The Deploy end- 
game is to know what to do when the time comes and 
to be able to move on it. 

You might wonder when you'll know that you’ve 
finished with the Deploy phase. That moment will 
come when you've done one end-to-end real-time exer- 
cise for an onsite and an offsite incident, using the pro- 
cesses you laid out in your written plan. Once you’ve 
tested your incident-response processes, conducted a 
performance review, documented and shared lessons 
learned, updated your written program, scheduled 
training for individuals who need remediation, and 
given your recovery program an “inaugural voyage,” 
you can move to day-to-day operations. 

A real disaster-recovery test can be hard for man- 
agers to approve. They'll fear the impact of being 
offline, the expense in time and hard costs, and the low 
payload. Perhaps a realistic starting point is a “table 
exercise” that walks through the plan within a set of 
scenarios. Perhaps your planned downtime activities 
can include smart testing of certain critical sections of 
your plan. Remember that an adequate test isn’t just 
about the technology working as promised. It should 
include activating replacement process participants 
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who might be performing a procedure they’ve never 
done in real life. The human element is seldom given 
the same resiliency review as the recovery technology. 


The Fifth D: Day-to-Day 
The final phase is program management, including 
periodic review, testing, and continuous refinement of 
recoverability using activity indictors. You'll find that 
day-to-day operation involves a fine balance of people 
and processes. Unlike automating a busines process, 
people can’t be automated out of the Deploy phase’s 
six core processes for disaster recovery. What to do? 
You need to balance process-management require- 
ments and people-management requirements. Process 
management focuses on traditional process execution 
and quality-management concepts (Plan/Do/Check/ 
Act), whereas people management focuses on individ- 
ual performer and team process execution concepts 
(Should/Can/Will/Did). “Should” is an agreement 
about what should happen, and it usually comes from 
the top. “Can” is proper resourcing. “Will” is depart- 
ment and individual motivation and commitment to 
perform. “Did” is performance evaluation and assess- 
ment, targeting continuous improvement. 


Final Checklist 

How do you know when you have a SQL Server disas- 

ter-recovery program that will work when you need 

it? Each organization has its own set of recoverability 
requirements, but it’s safe to say that if you’ve done 
the following, you're as prepared as you can be: 

e You've written a service level agreement (SLA) 
that guarantees a return to operations within so 
many hours or days, and you can abide by it. 

e You've assembled a team of experts, and you 
have access to talent for disaster-recovery plan 
writing, testing, and incident management. 

e You've written a well documented SQL Server 
disaster-recovery plan, and you test it annually as a 
real-time event and quarterly as a table-top exercise. 

e You have access to replacement systems and facilities. 

e You have confirmation that the disaster-recovery 
facility will be available as long as necessary until 
you can return to your original operations site. 


For more information about “must-haves,” see the 
“Baseline Checklist” sidebar. Remember, disaster- 
recovery programs include backups, but backups 
alone aren’t a program. Even if your data is recover- 
able, you won't have a SQL Server disaster-recovery 
program if all you have are backups. Finally, know 
that the program we’ve discussed is for your SQL 
Server environment; this isn’t an enterprise disaster- 
recovery program. If your company employs a 
business-continuity professional, be sure to make 
good use of this internal resource. SQL 

InstantDoc ID 128829 
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Using 


Table-Valued Parameters 
to Update Multiple Rows 


Update many values in thousands of records 


n previous versions of SQL Server, if I needed 

to update numerous rows of data, I used a 

temporary table. Suppose, for example, that 
thousands of transactions needed their transaction 
dates updated. I would upload the transaction IDs 
with the new date value in a temporary table, add a 
join between the transaction table and the temporary 
table, and update the transaction table with the new 
values from the temporary table. 

It seemed relatively simple to update thousands 
of records with temporary tables, but what if I had 
thousands of transactions that needed multiple field 
values updated? Or what if the updating became 
a regular request and the company needed to have 
the functionality available in the application? The 
process would be difficult to implement and would 
have a performance impact on the database. With 
table-valued parameters, available in SQL Server 
2008, the update process is easy to implement and 
has little performance impact on the database. Table- 
valued parameters can be passed as read-only input 
parameters. Because transaction logs aren’t recorded 
for table-valued parameters, performance improves. 
Previously, when multiple values were involved, the 
insert statement was executed multiple times; now, 
I use a table-valued parameter—with numerous 
values—that’s treated as one value and requires only 
one insert statement. The reduced number of trips 
between client and server boosts performance. 


Creating and Using Table- 
Valued Parameters in T-SQL 
A table-valued parameter is a new type of stored- 
procedure capability available in SQL Server 2008. 
Basically, a table-valued parameter lets you use an 
array of data in T-SQL, as well as send an entire 
data-set table as a parameter in a stored procedure or 
function. SQL Server 2008 stores this parameter as a 
user-defined table type. You can also add primary key 
and unique constraints to the table type. 

The ddl_script.sql in Listing 1 sets up the test data- 
base MovieRentalDB with a table and the table type. 
Additionally, it populates the table with values. 
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The three steps that follow, each with its code 
segment, let you see how to create and execute table- 
valued parameters in T-SQL. First, use the following 
code segment to create a new user-defined table 


type: 


--Create new table type to use as a 
table-value parameter 
CREATE TYPE dbo.ChangeTransDate AS 
TABLE( 
TransID varchar(36) NOT NULL, 
NewDate datetime NOT NULL) 


Matthew S. 
Harris 


(matthew.harris@ collegeminer.com) has more 
than eight years of experience with database 
development and administration. He is 
co-founder and Head of Technology at College 
Second, create a stored procedure with a table-valued Miner (www.collegeminer.com), a data mining 
parameter using Listing 2. Third, execute the stored 
procedure in Listing 3. Figure 1 shows the results of 
the insert statement in Listing 3. 

Note that storedprocedure_script.sql is the 
insert procedure that passes the table-valued 
parameter with the matthewharrisapplication. 
Book3.csv, which you can download from 
www.sqlmag.com, InstantDoc ID 128864, 
is an example comma-delimited file of the 
values that will be passed with the stored pro- 
cedure. Matthewharrisapplication is developed in C# 
and is also available for download. The application 
will work if the database is created on the local server. 
If the database is created on a different instance of 


and business intelligence company. He holds a 


Master of Science from Boston University. 


ORE on the WEB 


Download the code at 
InstantDoc ID 128864. 


LISTING |: Code to Populate 
MovieRentalDB with a Table and 
Table Type 


CREATE DATABASE [MovieRentalDB] 
USE MovieRentalDB 
CREATE TABLE TransLog( 
TransID int IDENTITY(1@1,1) PRIMARY KEY not null, 
UserID int not null, 
TransDate date not null, 
TypeID int not null) 
CREATE TYPE dbo.ChangeTransDate AS TABLE( 
TransID varchar(36) NOT NULL, 
NewDate datetime NOT NULL) 
INSERT INTO TransLog VALUES(1, '8/8/2009' ,1) 
INSERT INTO TransLog VALUES(2,'8/8/20809' ,2) 
INSERT INTO TransLog VALUES(3, '8/8/2099' , 3) 
INSERT INTO TransLog VALUES(4, '8/8/2009' ,4) 
INSERT INTO TransLog VALUES(5, '8/8/2899' ,5) 
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LISTING 2: Code to Create Stored Procedure 


--Create procedure to update many rows 

CREATE PROCEDURE [dbo]. [UpdateTransactionDates] 
@TableParam ChangeTransDate READONLY 

AS 


UPDATE b SET b.TransDate = a.NewDate 
FROM @TableParam a 
JOIN TransLog b ON b.TransID=a.TransID 


LISTING 3: Code to Execute Stored Procedure 


--Add the table-value parameter and call the stored procedure 
DECLARE @NewTableParam ChangeTransDate 

INSERT INTO @NewTableParam(TransID,NewDate) 

VALUES (101, '1/10/2989'), (102, '2/10/2999'), 

(103, '3/108/2089') , (184, '4/10/2909') , (105, '5/18/2099') 


EXEC dbo.UpdateTransactionDates @TableParam = @NewTableParam 


--Select from the table to verify the change 
SELECT * FROM TransLog 


(5 row(s) affected) 
TransID UserID TransDate TypeID 
101 I 2909-91-18 1 
192 2 2989-82-18 2 
103 3 2989-83-18 3 
104 4 2999-94-18 4 
105 5 2969-85-18 5 
\ 
Figure | 


Output from the Insert statement 


| C:\Users \mharis\Documents \Articles \MatthewHamisArticle\Book3.csv 


Figure 2 


Verifying DataViewGrid data 


1 

2 2 2 

a 103 3 2009-08-10 3 

4 104 4 2009-08-11 4 

5 105 5 2009-08-12 5 
Figure 3 


Verifying updated values 
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SQL Server than local server), SQLConnection will 
have to be changed. 


Adding the Functionality to a 
NET Application 

It would be difficult to add thousands of records to 
the table type with the code because I would have 
to use a T-SQL INSERT statement to insert each 
of the records. The easiest way that I’ve found to 
insert numerous rows into the table type is through 
a .NET application. At first, I thought using the 
.NET application would be difficult, but as you'll see, 
it involves few steps and very few lines of C# code. 
(I used .NET Framework 3.5, which isn’t necessary 
but is recommended.) I'll create an application that 
parses the data from a .csv file into a DataGridView, 
sets the DataGridView as a DataTable, and passes 
the DataTable as the table parameter. 

I then call the stored procedure to update the 
records. Note that I’ve updated about 10,000 records. 
Although the process is a little slow, it eliminates the 
time I would spend on the task, which is more valu- 
able to the company. 


Accessing Table-Valued 
Parameters Using a .NET 
Application 

Before you proceed to Web Listing 1 (www.sqlmag 
.com, InstantDoc ID 128864), a .NET applica- 
tion that accesses table-valued parameters, create 
a new Windows Form C# Application and drag a 
DataGridView onto the form (an OpenfileDialog), as 
well as two buttons (Browse and Update) and a label. 
Web Figure 1 shows the form. 

As you can see in Web Listing 1, using a .NET 
application makes it simple to access table-valued 
parameters. Callouts A through E highlight the steps 
that show you how to access table-valued parameters. 
(If you're new to application development, the callout 
steps will help get you started.) 

The code in callout A adds three assembly refer- 
ences. The code in callout B declares the DataTable 
and sets the DataGridView as this DataTable. In the 
namespace of your application, create a private data 
table. The code in callout C creates a parseCSV as a 
public list. 

The code in callout D adds functionality to the 
Browse button. Clicking Browse lets the user search 
for the .csv file with the records that need to be 
updated and add those records to the DataGridView. 
This segment of code calls parseCSV. The code in 
callout E adds functionality to the Update button. 
You want to create a SQL connection, call the stored 
procedure, and pass the DataGridView as the table 
parameter. 

The following brief procedure will help you test 
functionality. First, click Browse to find the .csv 
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file, as shown in Web Figure 2. Second, confirm 
that the text in DataGridView is correct, as Figure 
2 shows. 

Click Update (at this point, you can add some 
error handling). Finally, use the following code 
to verify that the values have been updated in the 
database: 


--Select all from updated table 
SELECT * 
FROM TransLog 


As Figure 3 shows, the values have been updated. As 
you can see, it’s quite straightforward to use a .NET 
application with table-valued parameters to update 
multiple field values in thousands of files. 


Before I used the user-defined table type, the 
stored procedure I relied on involved a cursor loop 
that took too long to execute, caused numerous 


deadlocks, and had a negative affect on database 
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performance. With SQL Server 2008’s new user- 
defined table type, you no longer have to spend 
hours trying to figure out ways to update thousands 
of records. You can also see how easy it is to add 
this functionality to an application; doing so lets 
users update the transactions themselves, rather 
than sending a request to the DBA. The user- 
defined table type lets you pass a table as a param- 
eter to a stored procedure to update records. Using 
it is simple, secure, and doesn’t negatively affect the 
performance of the database. SOL} 
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Securing 


Credit Card Data 
Through SQL Server 2008 


Take advantage of SQL Server features that can help 
your organization achieve PCI DSS compliance 


process credit card information, you're prob- 

ably aware that credit card fraud is a growing 
problem. These days, the theft of your physical 
credit by a robber is nothing compared to the mas- 
sive data breach of credit card information that 
occurs regularly. Malicious hackers have penetrated 
the electronic records of large organizations and 
acquired or exposed the credit card data of hun- 
dreds of millions of accounts. Data theft on this 
scale has cost these organizations—and the card 
providers—millions of dollars in lawsuits, fines, 
restitution for losses, and the expense of notifying 
account holders. 

To minimize the risk of storing credit card 
data on computers, major card providers American 
Express, Discover Financial Services, JCB, Master- 
Card Worldwide, and Visa have created the Payment 
Card Industry Security Standards Council (PCISSC), 
a self-policing industry association. In turn, the 
PCISSC has created the PCI Data Security Standard 
(PCI DSS) to guide merchants and service providers 
in safeguarding their credit card data. 

You might wonder whether businesses other than 
large banks and financial institutions have to pay 
attention to PCI DSS. The answer is that any business 
that uses or stores customer credit card data must 
become compliant with the PCI DSS. All organiza- 
tions that accept credit cards have direct mandates to 
comply with the standards as part of their business 
arrangements with the major credit card companies. 
In addition, such organizations must also do busi- 
ness only with other compliant organizations. This 
is a powerful incentive for organizations to become 
compliant if they provide services to businesses that 
accept credit cards. 

For many small merchants, the solution is rather 
simple: Outsource credit card transactions to a known 
compliant service provider, and retain no credit card 
information whatsoever. However, compliance with 
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PCI DSS is necessary for businesses that do handle 
credit card data for their own customer transac- 
tions or that provide services—such as processing or 
storage—that involve credit card data. 


PCI Data Security 
Requirements 

The PCISSC requires security of “cardholder data,” 
which it defines as the Primary Account Number 
(PAN) of the credit card with any of the following 
data: 

e Cardholder’s name 

e Expiration date of the credit card 

e Service code 


Edmund X. 
DeJesus 


(exdejesus@ gmail.com) is a freelance 


This is the information that your organization must 
protect. 

The PCI DSS includes the following 12 data secu- 
rity requirements that organizations must satisfy to 
comply with association standards: 

1. Install and maintain a firewall configura- 
tion to protect cardholder data. 

2. Bypass vendor-supplied defaults for system 
passwords and other security parameters. 

3. Protect stored cardholder data. 

4. Encrypt transmission of cardholder data 
across open, public networks. 

5. Use and regularly update antivirus 
software. 

6. Develop and maintain secure systems and 
applications. 

7. Restrict access to cardholder data by a 
business need-to-know basis. 

8. Assign a unique ID to each person that has 
computer access. 

9. Restrict physical access to cardholder data. 

10. Track and monitor all access to network 
resources and cardholder data. 

11. Regularly test security systems and 
processes. 
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12. Maintain a policy that addresses informa- 
tion security. 


Some of these requirements involve network and 
operational procedures that don’t directly affect the 
database. These are items 1, 5, 9, 11, and 12. However, 
the remaining requirements clearly apply to databases 
in which credit card data is stored or processed. 

In the remainder of this article, you'll learn how 
to apply the database-related PCI DSS data security 
requirements to your SQL Server database. 


Individual Requirements: 

A Closer Look 

The 12 PCI DSS requirements can seem fairly 
vague. For example, Requirement 3 directs organi- 
zations to “protect stored cardholder data.” Most 
organizations probably believe that they’re already 
doing this. To help organizations better understand 
the requirements, the PCISSC has issued “Navi- 
gating PCI DSS: Understanding the Intent of the 
Requirements” (www.pcisecuritystandards.org/pdfs/ 
pcei_dss_saq_navigating_dss.pdf), a document that 
explains the details of each requirement. For example, 
Requirement 3 actually consists of 20 subsections 
that describe how to satisfy the requirement. Some of 
these subsections are simple. For example, 3.2.2 states 
the following: “Do not store the card validation code 
or value (three-digit or four-digit number printed on 
the front or back of a payment card) used to verify 
card-not-present transactions.” Other subsections are 
quite complicated. For example, 3.1 states the follow- 
ing, in part: “Develop a data retention and disposal 
policy.” Clearly, this single statement could result in 
the creation of entirely new organizational policies, 
actions, and departments. 

Although you can perform all the work to 
achieve PCI DSS compliance, an independent audit 
is required to verify that your organization is, in fact, 
compliant. At some point, you must engage the ser- 
vices of a Qualified Security Assessor (QSA) to audit 
your organization according to the 12 requirements 
and their specific subsections. For this reason, you 
should make sure that you thoroughly document all 
the work that you do to satisfy the PCI DSS, and you 
should also document your operational procedures. 


Removing Defaults 
Requirement 2 specifies that you remove default 
aspects of computer systems, such as guest accounts. 
Malicious hackers often use default passwords, set- 
tings, and configurations to gain access to data or to 
enhance their credentials. By removing defaults, you 
deny malicious hackers easy access to the data. 

For example, one subsection of Requirement 2 
specifically mentions changing vendor-supplied pass- 
words. When you install SQL Server 2008, there are 


no default passwords; you must enter a new password 
when you create accounts. Additionally, SQL Server 
doesn’t permit easy-to-guess passwords, such as the 
name of the computer, the login name, null or empty 
passwords, or specific low-security passwords such 
as “admin,” “administrator,” “password,” “sa,” or 
“sysadmin.” 

Another subsection of Requirement 2 specifies 
disabling all unnecessary and insecure services and 
protocols. By default, many SQL Server 2008 features 
aren't active. These include the following features: 

e Ad Hoc Remote Queries 

e Change Data Capture (CDC) 

e CLR integration 

e Cross-database ownership chaining 
e Data Collector 

e Database Mail 

e Database mirroring 

e Dedicated administrator connection 
e HTTP connectivity 

e The .NET Framework 

e OLE Automation XPs 

e Remote connections 

e Service Broker 

e SQL Full-text Filter Daemon Launcher 
e SQLMail 

e Web Assistant 

e xp_cmdshell 
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If your installation of SQL Server requires any of 
these features, you can use SQL Server Management 
Studio (SSMS) or the sp_configure system stored pro- 
cedure to enable them. Similarly, if there are features 
that you know are unnecessary, you can use SSMS or 
sp_configure to disable them. 

SQL Server provides another important feature 
that’s disabled by default. During the application 
setup, you must select one of two authentication 
modes for the Database Engine: Windows Authenti- 
cation mode or mixed mode. If you select Windows 
Authentication mode, the sa account is disabled by 
default. 

The new SQL Server 2008 installations also 
don’t include the BUILTIN\Administrators local 
Windows group in the sysadmin fixed server role. 
And if any of your processes or code require access 
to the BUILTIN\Administrators group, you must 
explicitly grant permission to log in to SQL Server. 
The fact that an account is a member of the Admin- 
istrators group doesn’t automatically grant that 
account access to SQL Server. 

Using Windows Authentication to connect to 
SQL Server has several advantages for security. For 
example, you don’t have to worry about user cre- 
dentials passing over the network during authentica- 
tion. Potential attackers who monitor the network 
for usernames or passwords won't find anything. 
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Additionally, users must satisfy the security policies 
that have been established for Windows Authentica- 
tion, including minimum password lengths, password 
complexity requirements, mandatory password expi- 
ration times, and lockout after a specified number of 


incorrect login attempts. 


Windows Authentication also links a SQL Server 
identity to an Active Directory account. Using Active 
Directory Domain Services for access management 
enables SQL Server to enforce password policy from 
a central location, disable accounts, and maintain 


consistent user identities across all servers. 


If SQL Server does use Windows Authentication, 
it can support the Kerberos authentication protocol 
through the use of the Windows Security Support Pro- 
vider Interface. Kerberos provides mutual client-server 
authentication by using encrypted authentication tick- 
ets. These tickets let a logged-in user access any service 
without logging in again. It’s a best practice to use 


Kerberos authentication to connect to SQL Server. 


Protecting Cardholder Data 


Requirement 3 specifies protecting cardholder data in 
various ways. Some of this is as simple as not keep- 
ing cardholder data longer than necessary. But most 
of the subsections of Requirement 3 deal with the 
encryption of data. SQL Server 2008 offers several 
different features that help you encrypt data. These 


include the following features: 

e Transparent Data Encryption (TDE): Using 
a security-enhanced database encryption key 
(DEK) simplifies encryption at the database 


level. You can select and configure transparent 
encryption in SSMS. Using TDE involves the 


following steps: 


LISTING l: Encrypting and Decrypting the Sample 
Database 


USE master; 

GO 

CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<Password>'; 
GO 

CREATE CERTIFICATE CertificateServer WITH SUBJECT = 'Sample Certificate' 
GO 

USE Sample 

(60) 

CREATE DATABASE ENCRYPTION KEY 

WITH ALGORITHM = AES_128 

ENCRYPTION BY SERVER CERTIFICATE CertificateServer 

GO 

ALTER DATABASE Sample 

SET ENCRYPTION ON 

GO 


TABLE |:TDE Commands and Functions 
CREATE DATABASE ENCRYPTION KEY 
ALTER DATABASE ENCRYPTION KEY 
DROP DATABASE ENCRYPTION KEY 
ALTER DATABASE SET <Options> 


Creates a key to encrypt the database 
Changes the key to encrypt the database 
Removes the key that encrypted the database 


Explains the ALTER DATABASE option that's 
used to enable TDE 
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1. Create a master encryption key. 

2. Create or obtain a digital certificate pro- 
tected by the master encryption key. 

3. Create a database encryption key and use 
the digital certificate to protect the key. 

4. Set the database to use encryption. 


Listing 1 uses the commands and functions shown 
in Table 1 to encrypt and decrypt the Sample database 
by using a certificate that’s installed on the Certificate- 
Server server. (For more information, see “Securing 
Your SQL Server Environment,” www.sqimag.com, 
InstantDoc ID 102775.) 

e Cell-level encryption: Cell-level encryption 
was introduced in SQL Server 2005 and is still 
fully supported as a series of built-ins and as 
a key management hierarchy. However, this 
type of encryption doesn’t allow the use of 
automatic query optimization techniques. 

e Extensible Key Management (EKM) enables 
an external cryptographic provider, such as 
Hardware Security Module (HSM), to manage 
parts of the cryptographic key hierarchy. The 
cryptographic provider handles encryption 
and decryption operations by using these keys. 
The asymmetric keys provisioned by EKM are 
the only asymmetric keys that TDE supports. 

I strongly recommend that you use EKM 
together with both database-level and cell-level 
encryption for more comprehensive key man- 
agement and hardware-based cryptography if 
EKM is available through the HSM. You can 
enable EKM by using the sp_configure stored 
procedure. 


Another provision of Requirement 3 is to make 
sure that the whole PAN isn’t visible. The standard 
procedure is to mask the PAN when it’s displayed, 
except for either the first six digits or the last four 
digits. This is a task that any database (including 
SQL Server) can perform. One way to achieve 
this result is to create a field specifically to view 
the PAN. To do so, you can combine the first six 
digits, a string of symbols, and the last four digits. 
The resulting field is then used whenever a display 
is required. 


Encrypt Cardholder Data 
Across Public Networks 
Requirement 4 mainly involves protecting cardholder 
data that travels across public networks, including 
the Internet and wireless technologies. SQL Server 
has features to help meet this requirement, including 
support for Secure Sockets Layer (SSL) encryption. 
To use SSL encryption, you must install a certificate 
on the server by using the Microsoft Management 
Console (MMC) snap-in. 
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Develop Secure Applications 

This one seems obvious, right? Most of Requirement 
6 involves establishing processes for secure applica- 
tion development, including industry best practices, 
testing, validation, the use of security patches, the 
separation of development and production environ- 
ments, and the use of change control procedures. Web 
applications have their own set of rules. This creates a 
structure of secure development that has SQL Server 
as its target, not as its tool. 

It’s especially important that you apply all security 
patches to your installation. Surveys indicate that few 
database administrators ever apply security patches. 
However, PCI standards require that you apply criti- 
cal vendor-supplied security patches within a month 
of their release. 


Restrict Access to Cardholder 
Data 
On a fundamental level, you should locate your data- 
base in an internal network zone that isn’t accessible 
from untrusted networks. Additionally, one of the 
fundamental principles for securing cardholder data is 
to limit who has access to it. There are, in reality, very 
few roles in an organization that absolutely require 
access to cardholder data. Restricting who is given the 
privilege of accessing cardholder data has two steps: 
defining the cardholder data in such a way that it can 
be referenced separately from any other kind of data, 
and defining roles in such a way that access is denied 
by default and is granted only explicitly. This type of 
approach is known by several related terms: 

e Least privileges—Each role has only the least 
privileges necessary to fulfill that role and no 
more. 

e Need to know—Only those roles that abso- 
lutely must know the information are allowed 
to see and use it. 

e Deny all—Access is denied to all roles as a 
default and granted only to specific roles. 


Implied in these definitions is the idea that 
privileges are granted not to people but to roles. The 
access that’s required for each role in an organization 
is defined, and only those people who have a specific 
role receive the access that’s associated with that role. 
If a person changes roles, their access also changes. 
This approach is called role-based access control 
(RBAC). 

SQL Server offers a wide range of securables that 
you can specify, including databases, application roles, 
objects, roles, schemas, types, and users. For those 
securables, SQL Server has a very granular system of 
granting permissions. By using SQL Server, you can 
define specific actions that roles can perform, including 
selecting, modifying, executing, viewing, and control- 
ling securables. You can use SSMS to define column 
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permissions by role, and apply this feature to securing 
the specific columns that hold credit card information. 

In SQL Server 2008 and SQL Server 2005, signed 
modules help establish the security of execution by 
using digital certificates while maintaining simplicity 
of structure and not requiring extensive knowledge by 
users. A module can be to a stored procedure, a func- 
tion, a trigger, or an assembly. For example, access 
to view V might be mediated by stored procedure S. 
Instead of giving users direct permission to V, you can 


Any business that uses or 
stores customer credit card 


data must become compliant 


with the PCI DSS. 


use a signed module to allow access through S. To do 
so, you create a digital certificate, and then grant that 
certificate permission to access the table in V. Then, 
you sign S by using the digital certificate. Finally, 
you create a role that has permission to execute S. 
Thus, any user who has that role can execute S and 
use a SELECT statement in V. In this way, signed 
modules support segregation of duties, which is part 
of Requirement 7. 

Note that this method lets users gain access to V 
without granting them permissions to V. Additionally, 
users don’t have to know the structure of the arrange- 
ment between S, V, and the digital certificate. The 
ability to hide this relationship is an added security 
feature. At the same time, accessing V doesn’t require 
any extra knowledge on the part of the user. 


Unique IDs 

Using Windows Authentication together with SQL 

Server 2008 provides additional and significant 

security. This is certainly true for satisfying Require- 

ment 8, which includes 21 specific rules about how to 
manage logon credentials. Windows Authentication 
supports unique IDs for SQL Server. The correct 
configuration of Windows Authentication can satisfy 
the following specific requirements for your SQL 

Server installation: 

e 8.1: Assign all users a unique ID to access 
cardholder data. 

e 8.2: Employ at least a password or passphrase 
or two-factor authentication to authenticate 
users. 

e 8.4: Render all passwords unreadable dur- 
ing transmission and storage by using strong 
cryptography. 

e 8.5: Make sure that appropriate user authenti- 
cation and password management are used. 
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e 8.5.1: Control the addition, deletion, and 
modification of user IDs and credentials. 

e 8.5.3: Set first-time passwords to a unique 
value for each user and change the password 
immediately after the first use. 

e 8.5.5: Remove/disable inactive user accounts at 
least every 90 days. 

e 8.5.9: Change user passwords at least every 90 
days. 

e 8.5.10: Require a minimum password length 
of at least seven characters. 

e 8.5.11: Use passwords that contain both 
numeric and alphabetic characters. 

e 8.5.12: Don’t let an individual submit a new 
password that’s the same as any of the last 
four passwords used. 

e 8.5.13: Limit repeated access attempts by 
locking out the user ID after no more than six 
attempts. 

e 8.5.14: Set the lockout duration to a mini- 
mum of 30 minutes or until the administrator 
enables the user ID. 

e 8.5.15: If a session has been idle for more than 
15 minutes, require the user to re-enter the 
password to reactivate the terminal. 

e 8.5.16: Authenticate all access to any database 
that contains cardholder data. 


Clearly, using Windows Authentication can help 
secure SQL Server for PCI compliance. 


Track Access to Cardholder 
Data 

Requirement 10 specifies tracking any access to card- 
holder data and maintaining logs of all such access. 
In addition, those same logs must be safeguarded to 
prevent anyone from falsifying records to hide illegal 
access, 

SQL Server 2008 includes several features that 
provide or support such tracking capabilities. We have 
already discussed the use of RBAC to limit and define 
access. The use of RBAC also enables administrators 
to track and monitor what actions those roles—and, 
by extension, the people who fill those roles—are 
performing. 

Additionally, SQL Server Audit enables the cre- 
ation of automatic logging of defined events to a 
specified location. The events can include actions at 
the server level (e.g., login events, changes to manage- 
ment), at the database level, and in the auditing pro- 
cess itself. SQL Server Audit also defines “audit action 
groups.” These consist of actions that are conceptu- 
ally related and that can be tracked as a single entity. 
For example, the SUCCESSFUL_LOGIN_GROUP 
audit action group consists of several actions that are 
connected to successful logins to SQL Server, such 
as new connections or reused connections from a 


connection pool. At the database level, you can audit 

such actions as delete, execute, insert, receive, select, 

and update as performed on objects such as tables, 
views, and stored procedures. You can use either 

SSMS or T-SQL to create audits. The list of actions, 

action groups, and objects is extensive. (For more 

information, see the Related Articles box at www 

.sqimag.com, InstantDoc ID 128820.) 

The results of audit actions are sent to a specified 
target, such as a file, the Windows Security event log, 
or the Windows Application event log. The Windows 
Security event log is the preferred target because 
access to the Windows Application event log is pos- 
sible for any authenticated user. 

If you decide to use a file as a target, you should 
engage the following security measures to restrict 
access to this file: 

e Make sure that the SQL Server service 
account has read and write permissions for 
this file. 

e Make sure that audit administrators have read 
and write permissions for this file. 

e Encrypt the file and the folder that contain the 
file. 

e Define an audit for access to the file itself. 

e Generate reports on this file by using a differ- 
ent instance of SQL Server from the one that 
the log describes. 


You can also enable SQL Server C2 auditing. C2 is 
an NCSC security rating for software. C2 auditing 
monitors the actions of users who have administra- 
tive privileges, and it restricts access to logs. You can 
enable C2 auditing by using SSMS or the sqlemd 
command-line utility. 

SQL Server Policy-Based Management is an 
additional tool that helps administrators define and 
enforce rules for servers running SQL Server. The 
Policy-Based Management feature replaces many 
features of the Surface Area Configuration tool. Poli- 
cies can enforce naming conventions in a database or 
configuration options on multiple servers. This helps 
maintain consistency across servers. Policy-Based 
Management can also help maintain PCI-compliant 
configurations. 


Taking Security Seriously 

Keep in mind that it’s important to carefully record 
the processes that your organization uses to achieve 
compliance. Your records can help a QSA verify all 
the steps that have been performed. 

Achieving PCI DSS compliance is a complex pro- 
cess for an organization to achieve. Fortunately, SQL 
Server provides many features to help you achieve 
compliance. The result will be enhanced security for 
your entire organization. SQL 
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Fam 5 FAQs on SQL Server 


2008 R2 Parallel Data 


Warehouse 


hen Microsoft released the new SQL Server 

2008 R2 product line, it also introduced a new 
edition of SQL Server: SQL Server 2008 R2 Parallel 
Data Warehouse. The Parallel Data Warehouse Edition 
is quite different from other editions of SQL Server. 
Here are answers to the most frequently asked questions 
about the Parallel Data Warehouse Edition. 


What is SQL Server 2008 R2 
Parallel Data Warehouse? 

Formerly code-named Madison, SQL Server 2008 
R2 Parallel Data Warehouse is essentially the SQL 
Server equivalent to the Windows Server High Perfor- 
mance Computing (HPC) Edition. The Parallel Data 
Warehouse Edition will utilize Massively Parallel 
Processing (MPP) scale-out architecture to support 
very large data warehouses. It’s decidedly not a low- 
end offering—Microsoft is licensing it at $57,489 per 
processor. 


How is it different from other 
editions of SQL Server? 

Unlike the other editions of SQL Server, which are 
software-only products, the Parallel Data Warehouse 
Edition is a combination hardware and software 
offering. You can’t just buy SQL Server 2008 R2 
Parallel Data Warehouse Edition from Microsoft 
and install it on one of your own servers as you 
can with the other editions of SQL Server. Instead, 
you purchase SQL Server 2008 R2 Parallel Data 
Warehouse from one of Microsoft’s partners, which 
are HP, Dell, IBM, and Bull. Each of the different 
partners provides a variety of different sizes of system 
configurations. 


Is the Parallel Data Warehouse 
Edition the same thing as the 
Fast Track Data Warehouse? 

No. The Fast Track Data Warehouse and the Parallel 
Data Warehouse Edition are different. SQL Server Fast 
Track Data Warehouse is a set of reference hardware 


configurations that are available from several different 
Microsoft partners. These hardware configurations have 
been optimized by each of the vendors to provide excel- 
lent out-of-the-box performance for data warehousing 
implementations. However, Fast Track Data Warehouse 
system configurations are standard Symmetrical Mul- 
tiProcessing (SMP) systems that run the SQL Server 
Standard, Enterprise, or Datacenter Editions. 


Who is it targeted toward? 

It’s targeted toward organizations that have very large 
data warehousing requirements. Solutions from the 
different partners are designed to support data ware- 
houses ranging from 10TBs to more than 1PB. The 
Parallel Data Warehouse Edition is designed for busi- 
nesses whose data warehousing requirements push the 
limits of a standalone SQL Server implementation. 


How is it architected? 

With the other editions of SQL Server, queries are 
processed within one physical instance of a database 
using that system’s CPUs, memory, and storage. 
The Parallel Data Warehouse Edition’s “appliance” 
consists of a Control Node and multiple Compute 
Nodes. Tables are partitioned across multiple Com- 
pute Nodes where each node has its own CPU, 
memory, and storage. Since the data is distributed 
across multiple Compute Nodes, the Parallel Data 
Warehouse Edition is able to process queries in 
parallel where each node is acting on its own data. 
This enables the queries for these large tables to be 
completed much faster than processing them on a 
single SMP system. To add additional scalability to 
a Parallel Data Warehouse system, you simply add 
more Compute Nodes. 

You can find out more about the SQL Server 
2008 R2 Parallel Data Warehouse at InstantDoc ID 
125098 and from the Microsoft Parallel Data Ware- 
house website (www.microsoft.com/sqlserver/2008/en/ 
us/parallel-data-warehouse.aspx). SQL 
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